CENS Chromagrams: Robust Pitch-Class Analysis

• CENS chromagrams are pitch-class features that emphasize stable tonal content while suppressing transient dynamics and timbre variations.
• They are computed from a constant-Q transform chromagram followed by per-frame L1 normalization, temporal smoothing, downsampling, and coarse quantization.
• Although designed for robust harmonic analysis in music tasks, CENS underperform when used with deep CNNs for environmental sound classification compared to MFCCs and mel-scaled spectrograms.

Chroma Energy Normalized Statistics (CENS) chromagrams are pitch-class features designed to emphasize stable, longer-term tonal content while suppressing short-term variations due to dynamics, articulation, and timbre. They are derived from chroma features, which represent energy over 12 pitch classes modulo the octave, and are then processed by energy normalization, temporal smoothing, downsampling, and coarse quantization. In the ESC-50 environmental sound classification studies that evaluated mel-scaled spectrograms, mel-frequency cepstral coefficients (MFCCs), cyclic tempograms, short-time Fourier transform (STFT) chromagrams, constant-Q transform (CQT) chromagrams, and CENS chromagrams as 2D inputs to deep convolutional neural networks (CNNs), CENS were consistently the weakest-performing chroma variant and lagged substantially behind mel-scaled spectrograms and MFCCs (Wolf-Monheim, 2024, Wolf-Monheim, 9 Sep 2025).

1. Definition and representational objective

A chromagram is a time–pitch-class representation that aggregates spectral energy into 12 pitch classes, collapsing all octaves. In formal terms, for frame $n$ and pitch class $p$, a chroma bin $c_n(p)$ aggregates energy from all frequency bins mapped to $p$. CENS chromagrams extend this representation to obtain robustness to dynamics, timbre, and small timing mismatches through energy normalization and temporal “statistics,” namely smoothing and, in standard practice, quantization (Wolf-Monheim, 9 Sep 2025).

The studies describe CENS as a robust and scalable representation of tonal structures whose basis is CEN (Chroma Energy Normalization), followed by quantization using “log-like” amplitude thresholds. Their stated goal is to emphasize stable pitch-class patterns and de-emphasize transient details. Accordingly, CENS summarize which pitch classes are present and stable over time and are characterized as well-suited for music structure analysis, cover song identification, harmonic alignment, and similarity, rather than for tasks dominated by broadband, noise-like, or transient acoustic events (Wolf-Monheim, 2024).

This characterization is central to understanding later results. The same invariances that make CENS attractive for harmonic analysis also reduce sensitivity to short-lived local cues. The papers therefore treat CENS not as a generic spectral front end, but as a tonal descriptor with a specific inductive bias.

2. Computational pipeline

The reported CENS computation begins from a CQT chromagram extracted with librosa and follows Müller & Ewert’s Chroma Toolbox. The 2024 study explicitly states that the authors compute CENS from CQT chromagrams, then apply two additional steps: L1 normalization of each chromagram vector and quantization of amplitudes based on “log-like” amplitude thresholds. The 2025 study likewise states that CENS are derived from a CQT chromagram, followed by per-frame L1 energy normalization, temporal smoothing, and quantization, although exact parameter values are not provided (Wolf-Monheim, 2024, Wolf-Monheim, 9 Sep 2025).

The CQT and chroma mapping are given in the 2025 study as

$$X_{CQ}(m,k) = \sum_{n=0}^{N-1} x(n) g_k(n) e^{-j 2\pi k n / N}$$

where $g_k(n)$ are CQT basis filters, and

$C(m,p) = \sum_{k \in K(p)} W(p,k) |X_{CQ}(m,k)|$

where $K(p)$ are CQT bins mapped to pitch class $p$ and $W$ is a mapping weight. The paper then defines chromagram energy normalization (CEN) by per-frame L1 normalization:

$p$0

It further states that the “statistics” stage comprises quantization based on “log-like amplitude thresholds” and temporal smoothing, for example by a mean filter, to reduce noise and fluctuations (Wolf-Monheim, 9 Sep 2025).

The 2024 study provides a more explicit canonical reconstruction of the CENS pipeline as commonly used in the literature and in librosa’s implementation. Let $p$1 be the time-domain audio signal. After optional preprocessing such as resampling and mono conversion, the constant-Q transform is denoted by $p$2, and chroma aggregation over pitch classes $p$3 is written as

$p$4

Per-frame L1 normalization is then

$p$5

with $p$6 and $p$7 a small constant to avoid division by zero. Temporal smoothing is expressed as

$p$8

followed by downsampling,

$p$9

and coarse quantization with thresholds $c_n(p)$0 mapped to $c_n(p)$1 (Wolf-Monheim, 2024).

Both studies are explicit that important implementation details are unreported. The papers do not fully specify sampling rate handling, frame size, hop size, tuning reference, window type, compression exponent, normalization epsilon, smoothing kernel type and length, downsampling factor, or the exact numerical quantization thresholds used in the experiments. The 2025 study reports only that all features were generated using librosa and that CENS followed the Chroma Toolbox; the 2024 study similarly notes that the canonical formulas presented beyond L1 normalization and “log-like” thresholding are literature-based rather than directly enumerated in the paper (Wolf-Monheim, 2024, Wolf-Monheim, 9 Sep 2025).

3. Relationship to STFT and CQT chromagrams

The two studies compare CENS with STFT chromagrams and CQT chromagrams to clarify what information is retained or discarded at each stage. STFT chromagrams are computed from short-time Fourier transform magnitudes with uniform time resolution and linear frequency spacing. The 2025 study gives

$c_n(p)$2

and then

$c_n(p)$3

These features capture instantaneous pitch-class energy at relatively fine temporal resolution and are useful when transient pitch content or broadband energy must be localized in time (Wolf-Monheim, 9 Sep 2025).

CQT chromagrams use a logarithmic frequency resolution aligned more naturally with musical pitch classes. The time resolution increases with frequency, and the representation is generally better suited to harmonic analysis than STFT-based chroma. In both studies, CENS begins from this CQT-derived chroma, then applies energy normalization together with temporal smoothing and quantization (Wolf-Monheim, 2024).

The contrast is therefore not merely one of front-end transform. CENS trades time resolution and fine spectral detail for invariance. The papers describe it as emphasizing stable harmonic patterns while de-emphasizing short-term variability. This trade-off is beneficial for tasks requiring tonal robustness, but the same mechanism suppresses precisely the transient and non-harmonic detail that environmental sound recognition often requires.

4. Integration into deep convolutional neural networks

In both studies, CENS chromagrams are treated as 2D “image-like” inputs to a deep CNN. The exact tensor shape is not reported, but the same network is applied across all investigated feature types, including mel-scaled spectrograms, MFCCs, cyclic tempograms, STFT chromagrams, CQT chromagrams, and CENS chromagrams. A batch normalization layer “to normalize the spectrograms on the frequency axis” precedes the first convolutional layer, and the same architecture is used for the chromagram inputs as well (Wolf-Monheim, 2024).

The reported CNN architecture is:

• Batch normalization
• Conv2D with 64 filters, kernel size $c_n(p)$4, ReLU, same padding
• MaxPool2D with pool size $c_n(p)$5
• Conv2D with 128 filters, kernel size $c_n(p)$6, ReLU, same padding
• MaxPool2D with pool size $c_n(p)$7
• Conv2D with 256 filters, kernel size $c_n(p)$8, ReLU, same padding
• MaxPool2D with pool size $c_n(p)$9
• Conv2D with 256 filters, kernel size $p$0, ReLU, same padding
• MaxPool2D with pool size $p$1
• Flatten
• Dense with 256 units and ReLU
• Dropout with rate $p$2
• Dense with 50 units and softmax for ESC-50 classes (Wolf-Monheim, 2024, Wolf-Monheim, 9 Sep 2025)

The studies differ in how much training detail is reported. The 2024 study specifies optimizer, loss, callbacks, and an 80/20 random split of the 2,000 ESC-50 files into 1,600 training and 400 validation examples, but does not report learning rate, batch size, or input dimensions. The 2025 study provides a more explicit regimen: an 80%/20% stratified train/validation split, Adam with initial learning rate $p$3, sparse categorical cross-entropy, batch size 32, ReduceLROnPlateau with factor 0.1 and patience 2 on validation-loss plateau, EarlyStopping with patience 6 on validation loss, and no separate test set, with validation used as a proxy for testing (Wolf-Monheim, 2024, Wolf-Monheim, 9 Sep 2025).

The dataset context is also relevant. ESC-50 consists of 2,000 labeled environmental audio clips, 50 classes grouped into 5 categories, 40 clips per class, and each clip is 5 seconds long. The 2025 study states a sampling rate of 44.1 kHz and 16-bit audio. No feature-specific augmentation or task-specific tuning for CENS is described (Wolf-Monheim, 9 Sep 2025).

5. Reported empirical performance

The strongest empirical result associated with CENS in these studies is negative: CENS chromagrams produce the lowest validation accuracy among the tested feature sets in both evaluations. The 2024 paper reports training accuracy of 70.06%, validation accuracy of 11.50%, training loss of 0.96, validation loss of 4.76, and 21 epochs for CENS. The 2025 follow-up reports training loss of 1.535, training accuracy of 54.8%, validation loss of 3.953, and validation accuracy of 14.0% for CENS (Wolf-Monheim, 2024, Wolf-Monheim, 9 Sep 2025).

Feature	2024 validation accuracy	2025 validation accuracy
Mel-scaled spectrograms	57.50%	61.8%
MFCCs	56.00%	58.8%
Cyclic tempograms	22.25%	23.3%
STFT chromagrams	28.25%	21.5%
CQT chromagrams	19.25%	21.3%
CENS chromagrams	11.50%	14.0%

The broader pattern is consistent across both papers: mel-scaled spectrograms and MFCCs perform substantially better than the chromagram-based representations and cyclic tempograms for environmental sound classification. In the 2025 study, category-level means for chromagram-based features collectively fall within approximate ranges of 36.2%–45.6% accuracy, 36.3%–46.5% precision, 35.7%–46.5% recall, and 35.9%–46.4% F1, while class-level mean precision for chromagram-based features is reported as 18.4% over the 50 classes. The paper does not break out category-level or class-level precision figures specifically for CENS, but it identifies chromagram-based features collectively as lagging behind mel-scaled spectrograms and MFCCs (Wolf-Monheim, 9 Sep 2025).

Both studies also delimit the statistical strength of these results. The 2024 paper presents single-split results and does not report variability across folds or repeated runs, nor formal significance testing. It notes that standard ESC-50 practice is 5-fold cross-validation and that the absence of multi-fold metrics means statistical significance cannot be established from the reported numbers. The 2025 paper similarly uses a single train/validation split and no separate test set (Wolf-Monheim, 2024, Wolf-Monheim, 9 Sep 2025).

6. Interpretation, limitations, and appropriate use

The papers attribute CENS underperformance on ESC-50 to a mismatch between feature design and task structure. ESC-50 contains many classes dominated by broadband noise, transients, and non-tonal events, including examples such as “rain,” “glass breaking,” “chainsaw,” and “clock alarm.” CENS, by design, emphasize harmonic pitch classes and stable tonal content, while temporal smoothing and quantization suppress transient energy and fine temporal detail. The 2024 study argues that when underlying signals are non-harmonic, the feature becomes sparse and less discriminative; the 2025 study similarly states that many environmental sounds are broadband, noise-like, or transient, with weak or no harmonic structures, which reduces the discriminative power of a pitch-centric representation (Wolf-Monheim, 2024, Wolf-Monheim, 9 Sep 2025).

A further issue identified in the 2025 study is that amplitude normalization removes loudness variations that may carry useful class-specific information in environmental sounds. The same study also notes that some classes have overlapping spectral properties or low signal-to-noise ratio, and that CENS’ tonal robustness does not counter these challenges effectively. The large train–validation gap reported for CENS in both studies reinforces the conclusion that the learned representation generalizes poorly in this setting (Wolf-Monheim, 9 Sep 2025).

The strengths of CENS remain domain-specific rather than universal. Both studies identify music-related tasks as appropriate use cases: key or chord recognition, cover song retrieval, structure analysis, harmonic alignment, and related settings in which tonal stability and invariance to dynamics or timbre are desirable. For general environmental sound classification, both papers recommend mel-scaled spectrograms and MFCCs as stronger baselines and suggest that, if chroma must be used, STFT or CQT chromagrams without heavy smoothing or quantization may preserve more task-relevant detail. They also propose hybrid strategies such as combining CENS with mel or MFCC features in a multi-branch CNN to capture both tonal and non-tonal cues (Wolf-Monheim, 2024, Wolf-Monheim, 9 Sep 2025).

The reported limitations are primarily methodological. The studies do not provide exhaustive CENS parameterization, do not conduct parameter sweeps or ablations, and do not perform standard ESC-50 5-fold cross-validation. The 2024 paper notes that hyperparameters such as input tensor dimensions, sampling rate, window sizes, smoothing length, quantization thresholds, batch size, and learning rate are not fully reported and that no code repository is provided. The 2025 paper similarly identifies the lack of explicit CENS parameterization, the single architecture, and the single train/validation split as constraints on reproducibility and interpretation. Suggested future directions include tuning smoothing window length and quantization thresholds, exploring L1 versus L2 normalization, omitting quantization, using event-synchronous aggregation where applicable, evaluating augmentation, and testing larger datasets or hybrid architectures (Wolf-Monheim, 2024, Wolf-Monheim, 9 Sep 2025).