Pesto in Pitch Estimation & Cryptography

• Pesto is a research topic with dual identities: one focusing on self-supervised pitch estimation using transposition equivariance and the other on a multivariate cryptographic scheme employing CCZ transformations.
• In pitch estimation, PESTO leverages Siamese networks with lightweight encoders and Toeplitz layers to enforce translation invariance, achieving low-latency and high accuracy on diverse audio benchmarks.
• In cryptography, Pesto utilizes a CCZ-based public-key construction to obscure quadratic trapdoors, though analysis reveals that the quartic appearance does not significantly enhance security.

Searching arXiv for papers on PESTO in pitch estimation and cryptography. Tool unavailable in this environment; proceeding with the arXiv records explicitly provided: (Riou et al., 2023, Riou et al., 2 Aug 2025), and (Caminata et al., 21 Jul 2025). Pesto denotes two distinct research entities in the arXiv literature. In music information retrieval, PESTO abbreviates Pitch Estimation with Self-supervised Transposition-equivariant Objective, a self-supervised framework for single-pitch estimation that exploits pitch-shift equivariance in log-frequency representations and is designed for lightweight, real-time deployment (Riou et al., 2023). A later real-time formulation retains the same acronym while using a streamable Variable-$Q$ Transform frontend, reporting low latency and strong cross-dataset generalization on music and speech benchmarks (Riou et al., 2 Aug 2025). In multivariate cryptography, Pesto names a CCZ-based public-key construction whose public key consists of dense degree-4 polynomials derived from a quadratic central map; a subsequent cryptanalysis argues that this quartic disguise can be reduced in polynomial time to an equivalent quadratic system (Caminata et al., 21 Jul 2025).

1. PESTO in self-supervised pitch estimation

The pitch-estimation line of work is motivated by two observations. First, many deep-learning pitch trackers depend on large labeled datasets or costly signal-processing pipelines, which limits deployment on resource-limited devices and can impair generalization across tasks. Second, in musical audio, pitch transpositions correspond to rigid shifts in a log-frequency representation such as the Constant-Q Transform, suggesting that a model constrained to be equivariant under transposition can learn pitch information without labeled data (Riou et al., 2023).

The 2023 formulation uses a Siamese self-supervised paradigm in which two differently transposed views of the same monophonic audio frame are processed by a shared lightweight encoder. The central requirement is that shifting the input by $k$ semitones should shift the network output by the same amount. The model was designed around three explicit goals: model complexity below $30{,}000$ parameters, no requirement for labeled data, and strong cross-dataset generalization between singing voice and instrumental recordings (Riou et al., 2023).

The 2025 formulation preserves the same conceptual core but reframes the problem as real-time single-pitch ($F_0$) estimation under causal, low-latency constraints. It emphasizes that supervised approaches such as CREPE require large annotated datasets and often fail to generalize to new instruments or domains, whereas classical DSP approaches such as YIN and SWIPE are lightweight but brittle under noise or timbre changes. PESTO’s stated objective in that version is to train a small neural estimator in a self-supervised way by exploiting the fact that pitch shifts correspond to frequency-axis translations in a log-frequency representation (Riou et al., 2 Aug 2025).

This suggests that the pitch-estimation meaning of PESTO is best understood as a family of transposition-equivariant SSL methods rather than a single frozen architecture: the 2023 paper establishes the principle with a Constant-$Q$ frontend and a $28.9$k-parameter encoder, while the 2025 paper extends the same principle to a streamable VQT-based real-time system with approximately $130$k parameters.

2. Signal representation and equivariant preprocessing

In the 2023 system, each audio frame $x$ is represented by a single-frame Constant-$Q$ Transform computed on the GPU via nnAudio. The minimum frequency is set to $f_{\min}=27.5$ Hz, corresponding to $k$0, and the representation uses $k$1 bins per semitone, for a total of $k$2 log-frequency bins for a $k$3 kHz signal (Riou et al., 2023).

Pitch shifting is simulated directly in the transform domain rather than with a phase-vocoder. For $k$4 with $k$5, the method crops shifted slices of the CQT frame: $k$6

$k$7

Both cropped views have the same length $k$8. In addition to transposition, pitch-preserving augmentations are applied, including additive white noise with standard deviation uniformly drawn in $k$9 and random gain in $30{,}000$0 dB (Riou et al., 2023).

The 2025 real-time formulation replaces CQT with a Variable-$30{,}000$1 Transform. The VQT is used because it provides a log-frequency axis where semitone shifts correspond to uniform bin translations. The stated parameters, unless otherwise noted, are sampling rate $30{,}000$2 such as $30{,}000$3 kHz, bins per semitone $30{,}000$4, minimum frequency $30{,}000$5 Hz, number of frequency bins $30{,}000$6, maximum pitch shift in training $30{,}000$7 bins, and VQT parameter $30{,}000$8 (Riou et al., 2 Aug 2025).

The VQT filter length is given by

$30{,}000$9

with $F_0$0 as stated in the source description (Riou et al., 2 Aug 2025). Pitch shifts are again simulated by cropping sub-frames: from an $F_0$1-bin frame $F_0$2, one extracts two $F_0$3 length sub-frames offset by $F_0$4, approximating a $F_0$5-bin pitch shift.

A common thread across both versions is that the data representation is not merely a frontend choice but part of the learning prior: the log-frequency axis turns pitch transposition into translation, making equivariance operationally enforceable.

3. Architecture and transposition-equivariant objective

The 2023 encoder is a shared Siamese network with only $F_0$6k parameters. It processes each CQT frame independently through LayerNorm, two parallel $F_0$7D convolutional layers with kernel size $F_0$8 and padding $F_0$9 plus a residual skip connection, then four additional $Q$0D convolutional layers with kernel size $Q$1, padding $Q$2, LeakyReLU with slope $Q$3, and dropout rate $Q$4. A final Toeplitz fully-connected layer followed by softmax maps the representation to a probability distribution over $Q$5 quantized pitches (Riou et al., 2023).

Toeplitz fully-connected layers are introduced to enforce exact transposition equivariance. The weight matrix $Q$6 satisfies

$Q$7

so that shifting the input vector results in a corresponding shift of the output. A Toeplitz matrix contains only $Q$8 parameters instead of $Q$9, and in the 2023 configuration the final layer therefore contributes only about $28.9$0 parameters (Riou et al., 2023).

The self-supervised objective is formulated on softmax-normalized outputs

$28.9$1

The group action $28.9$2 is a discrete shift of the output index, and equivariance requires

$28.9$3

equivalently

$28.9$4

To enforce this, the method combines three loss terms: an equivariance loss $28.9$5 based on a fixed linear projector $28.9$6 with $28.9$7, a shifted cross-entropy regularizer $28.9$8, and an invariance loss $28.9$9 between pitch-preserving augmentations. The full loss is

$130$0

with the weights tuned automatically via gradient-norm balancing (Riou et al., 2023).

The 2025 system retains the same high-level structure but scales the backbone. Its single-branch network $130$1 uses seven $130$2D convolutions along log-frequency, kernel size $130$3, three residual skip-connections, LeakyReLU with slope $130$4, dropout $130$5, flattening, and a final Toeplitz fully-connected layer. The Toeplitz layer is described as equivalent to a $130$6D convolution with padding $130$7, thereby preserving exact translation equivariance in the final mapping (Riou et al., 2 Aug 2025).

In both papers, collapse prevention is a central design constraint. The 2023 description states that no negative sampling is needed because equivariance losses inherently prevent collapse, while the 2025 description uses a full symmetric objective with gradient stops on targets and dynamic loss weighting based on the gradient norms at the last layer (Riou et al., 2023, Riou et al., 2 Aug 2025). A plausible implication is that the method differs from contrastive SSL not only in the absence of labels but also in the absence of explicit negatives, replacing instance discrimination with a structured group-action constraint.

4. Training, evaluation, and real-time operation

The 2023 training procedure computes CQT frames of length $130$8, crops them to length $130$9, uses batch size $x$0, and optimizes with Adam at learning rate $x$1, default $x$2-parameters, for $x$3 epochs with a cosine-annealing schedule. For each CQT frame, a shift $x$4 is sampled along with two random pitch-preserving augmentations (Riou et al., 2023).

Evaluation in that paper uses two monophonic pitch-estimation benchmarks: MIR-1K for singing voices and MDB-stem-synth for isolated instrument notes. Metrics are Raw Pitch Accuracy, defined as the percentage of voiced frames with error less than $x$5 semitone, and Raw Chroma Accuracy, which ignores octave errors (Riou et al., 2023).

For clean signals, the reported RPA results are:

Model	MIR-1K	MDB
SPICE	90.6%	89.1%
DDSP-inv	91.8%	88.5%
CREPE	97.8%	96.6%
PESTO trained on MIR-1K	96.1%	94.6%
PESTO trained on MDB	93.5%	95.5%

The same paper reports robustness experiments on MIR-1K with background music. A PESTO model trained on clean data drops from $x$6 on clean signals to $x$7 at $x$8 dB vocal-to-background SNR, whereas a PESTO model trained with background-mix augmentation retains approximately $x$9 at $Q$0 dB and clearly outperforms SPICE in noisy conditions (Riou et al., 2023).

The 2025 real-time study evaluates on MIR-1K, MDB-stem-synth, and PTDB. In self-supervised comparison, PESTO with $Q$1k parameters reports $Q$2 RPA on MIR-1K and $Q$3 on MDB-ss, versus SPICE at $Q$4 and $Q$5, and DDSP-inv at $Q$6 and $Q$7 (Riou et al., 2 Aug 2025). Cross-dataset results include PESTO trained on MIR-1K and tested on PTDB at $Q$8 RPA versus CREPE at $Q$9, and PESTO trained on PTDB and tested on MDB-ss at $f_{\min}=27.5$0 RPA versus PENN at $f_{\min}=27.5$1 (Riou et al., 2 Aug 2025). Multi-dataset training is reported at $f_{\min}=27.5$2 RPA on MIR-1K/MDB.

Real-time deployment is enabled by a streamable VQT implementation based on nnAudio but replacing standard convolutions with cached convolutions and a circular memory buffer. The naïve VQT latency is

$f_{\min}=27.5$3

where $f_{\min}=27.5$4 is the largest filter length and $f_{\min}=27.5$5 is the model forward time. The paper then introduces “buffer refilling,” yielding

$f_{\min}=27.5$6

With $f_{\min}=27.5$7, total lag is approximately $f_{\min}=27.5$8, and the combined system is reported to have total latency less than $f_{\min}=27.5$9 ms (Riou et al., 2 Aug 2025).

The reported Real-Time Factor values are $k$00 on CPU using an i9-12900H, $k$01 on GPU using an RTX A2000, and $k$02 for a YIN baseline on CPU (Riou et al., 2 Aug 2025). This supports the claim that PESTO is suitable for embedded and live applications.

5. The multivariate cryptographic scheme named Pesto

A separate research line uses the name Pesto for a multivariate public-key construction introduced by Calderini, Caminata, and Villa and analyzed in a later cryptanalytic work (Caminata et al., 21 Jul 2025). In that scheme, the public polynomials are obtained by applying a CCZ transformation to a set of quadratic secret polynomials, with the consequence that the public key consists of degree-$k$03 polynomials (Caminata et al., 21 Jul 2025).

The secret map is defined over a base field $k$04 with positive integers $k$05 satisfying $k$06 and $k$07. The $k$08 input variables are split as

$k$09

The central map $k$10 is

$k$11

where $k$12 and $k$13, with each $k$14 a random quadratic form in the $k$15-variables. The second block $k$16 is an Oil-Vinegar system of $k$17 quadratics with vinegar variables $k$18 and oil variables $k$19 (Caminata et al., 21 Jul 2025).

The scheme does not publish $k$20 directly. Instead, it samples affine-linear bijections

$k$21

defines a twisted secret map

$k$22

with $k$23, and publishes

$k$24

Since the first $k$25 components of $k$26 have degree at most $k$27 and the remaining ones degree at most $k$28, each public component $k$29 is a polynomial of total degree at most $k$30 in the public variables $k$31 (Caminata et al., 21 Jul 2025).

The intended significance of the design is that a quadratic central map is hidden behind a quartic public representation. This suggests a degree-based obfuscation strategy: the CCZ transform is used not to alter the underlying algebraic structure of the trapdoor, but to disguise it through a higher public degree.

6. Cryptanalysis and security implications

The cryptanalytic result argues that the degree-$k$32 public system can be efficiently reduced to a system of quadratic polynomials, which in turn suggests that the CCZ transformation does not offer a significant increase in security (Caminata et al., 21 Jul 2025). Two polynomial-time reduction methods are described.

The first method is a Grӧbner-basis-style elimination using a Macaulay matrix. It forms the vector space $k$33 of degree-$k$34 linear combinations of the public polynomials $k$35, then constructs the full Macaulay matrix in degree $k$36, performs Gaussian elimination augmented by the “mutant trick,” and shows that rows of final degree $k$37 or $k$38 are congruent modulo $k$39 to quadratic polynomials. The resulting system has $k$40 pure quadratics with the same common zeros as the quartic public equations. The total cost is stated as

$k$41

field operations (Caminata et al., 21 Jul 2025).

The second method is a Higher-Order Linearization Equation attack. It introduces the algebra of symbolic input/output monomials in variables $k$42, samples roughly $k$43 input guesses $k$44, computes the corresponding outputs $k$45, stacks the sampled monomial evaluations into a linear system, and extracts from the nullspace at least $k$46 linearly independent quadratic relations $k$47. Substituting $k$48 then recovers purely quadratic equations in $k$49 alone. The stated complexity is

$k$50

which becomes $k$51 in the regime $k$52 (Caminata et al., 21 Jul 2025).

The cryptanalysis provides numerical rough-order estimates: for $k$53, HOLE costs on the order of $k$54 field operations, and for $k$55, approximately $k$56 operations (Caminata et al., 21 Jul 2025). It concludes that Pesto as proposed in [CCV24] is no harder than solving an unbalanced OV system in $k$57 variables and $k$58 equations, that applying a CCZ transform to any central quadratic map admitting a $k$59-twist cannot yield super-quadratic security, and that CCZ offers no meaningful extra layer of security in multivariate-public-key schemes based on low-degree central maps (Caminata et al., 21 Jul 2025).

A common misconception in this setting is that quartic public equations necessarily imply quartic hardness. The cryptanalytic result directly contests that interpretation: the public degree is shown not to be a stable indicator of effective algebraic difficulty for this construction.

7. Comparative significance of the two usages

The two meanings of Pesto are unrelated in domain and methodology. The pitch-estimation PESTO is a self-supervised neural method for monophonic or single-pitch estimation, built around translation equivariance in log-frequency representations, Siamese training, Toeplitz layers, and real-time streaming constraints (Riou et al., 2023, Riou et al., 2 Aug 2025). The cryptographic Pesto is a multivariate public-key design built from a quadratic central map, an Oil-Vinegar component, and a CCZ transformation intended to yield a quartic public system (Caminata et al., 21 Jul 2025).

Despite the nominal overlap, the acronymic pitch-estimation literature and the cryptographic scheme employ distinct technical notions of equivariance and transformation. In the audio setting, equivariance is a learning prior: a pitch shift in the input should induce an equal shift in the output distribution. In the cryptographic setting, the relevant transformation is a structural disguise mechanism for public polynomials. The former uses Toeplitz operators to preserve translation symmetry by design; the latter uses affine bijections and CCZ-style composition to obscure an underlying trapdoor map.

The juxtaposition is instructive. In audio MIR, the central claim is that explicit structural constraints can improve sample efficiency, cross-dataset generalization, and real-time deployability without annotated data. In multivariate cryptography, the central claim of the cryptanalysis is nearly the opposite in spirit: a structural transformation intended to increase hardness may preserve enough algebraic regularity to be inverted by polynomial-time linear-algebraic methods. This suggests that “structure” is neither intrinsically beneficial nor intrinsically harmful; its effect depends on whether the application seeks invariance and generalization, as in SSL pitch estimation, or resistance to algebraic reduction, as in public-key design.