---
title: Pesto in Pitch Estimation & Cryptography
url: https://www.emergentmind.com/topics/pesto
type: topic
---

# Pesto in Pitch Estimation & Cryptography

Searching arXiv for recent papers on PESTO in pitch estimation and cryptography.
Tool unavailable in this environment; proceeding with the arXiv records explicitly provided: [2309.02265], [2508.01488], and [2507.15449].
Pesto denotes two distinct research entities in the arXiv literature. In music information retrieval, **PESTO** abbreviates **Pitch Estimation with Self-supervised Transposition-equivariant Objective**, a self-supervised framework for single-pitch estimation that exploits pitch-shift equivariance in log-frequency representations and is designed for lightweight, real-time deployment [2309.02265]. A later real-time formulation retains the same acronym while using a streamable Variable-\(Q\) Transform frontend, reporting low latency and strong cross-dataset generalization on music and speech benchmarks [2508.01488]. In multivariate cryptography, **Pesto** names a CCZ-based public-key construction whose public key consists of dense degree-4 polynomials derived from a quadratic central map; a subsequent cryptanalysis argues that this quartic disguise can be reduced in polynomial time to an equivalent quadratic system [2507.15449].

## 1. PESTO in self-supervised pitch estimation

The pitch-estimation line of work is motivated by two observations. First, many deep-learning pitch trackers depend on large labeled datasets or costly signal-processing pipelines, which limits deployment on resource-limited devices and can impair generalization across tasks. Second, in musical audio, pitch transpositions correspond to rigid shifts in a log-frequency representation such as the Constant-Q Transform, suggesting that a model constrained to be equivariant under transposition can learn pitch information without labeled data [2309.02265].

The 2023 formulation uses a Siamese self-supervised paradigm in which two differently transposed views of the same monophonic audio frame are processed by a shared lightweight encoder. The central requirement is that shifting the input by \(k\) semitones should shift the network output by the same amount. The model was designed around three explicit goals: model complexity below \(30{,}000\) parameters, no requirement for labeled data, and strong cross-dataset generalization between singing voice and instrumental recordings [2309.02265].

The 2025 formulation preserves the same conceptual core but reframes the problem as **real-time single-pitch (\(F_0\)) estimation** under causal, low-latency constraints. It emphasizes that supervised approaches such as CREPE require large annotated datasets and often fail to generalize to new instruments or domains, whereas classical DSP approaches such as YIN and SWIPE are lightweight but brittle under noise or timbre changes. PESTO’s stated objective in that version is to train a small neural estimator in a self-supervised way by exploiting the fact that pitch shifts correspond to frequency-axis translations in a log-frequency representation [2508.01488].

This suggests that the pitch-estimation meaning of PESTO is best understood as a family of transposition-equivariant SSL methods rather than a single frozen architecture: the 2023 paper establishes the principle with a Constant-\(Q\) frontend and a \(28.9\)k-parameter encoder, while the 2025 paper extends the same principle to a streamable VQT-based real-time system with approximately \(130\)k parameters.

## 2. Signal representation and equivariant preprocessing

In the 2023 system, each audio frame \(x\) is represented by a single-frame Constant-\(Q\) Transform computed on the GPU via nnAudio. The minimum frequency is set to \(f_{\min}=27.5\) Hz, corresponding to \(A0\), and the representation uses \(b=3\) bins per semitone, for a total of \(K=99\times b=297\) log-frequency bins for a \(16\) kHz signal [2309.02265].

Pitch shifting is simulated directly in the transform domain rather than with a phase-vocoder. For \(k\in[-k_{\max},k_{\max}]\) with \(k_{\max}=16\), the method crops shifted slices of the CQT frame:
\[
x = \mathrm{CQT}[k_{\max}:K-k_{\max}-1],
\]
\[
x^{(k)}=\mathrm{CQT}[k_{\max}-k:K-k_{\max}-1-k].
\]
Both cropped views have the same length \(N=K-2k_{\max}\). In addition to transposition, pitch-preserving augmentations are applied, including additive white noise with standard deviation uniformly drawn in \([0.1,2]\) and random gain in \([-6,+3]\) dB [2309.02265].

The 2025 real-time formulation replaces CQT with a Variable-\(Q\) Transform. The VQT is used because it provides a log-frequency axis where semitone shifts correspond to uniform bin translations. The stated parameters, unless otherwise noted, are sampling rate \(f_s\) such as \(16\) kHz, bins per semitone \(B=3\), minimum frequency \(f_{\min}=27.5\) Hz, number of frequency bins \(F\approx 99\cdot B\), maximum pitch shift in training \(k_{\max}=16\) bins, and VQT parameter \(\gamma=7\) [2508.01488].

The VQT filter length is given by
\[
w_k=\Bigl\lceil \frac{Q\,f_s}{f_k+\frac{\gamma}{\zeta}} \Bigr\rceil,
\]
with \(\zeta=2^{\tfrac{1}{12B}-1}\) as stated in the source description [2508.01488]. Pitch shifts are again simulated by cropping sub-frames: from an \(F\)-bin frame \(x\), one extracts two \(F'=F-2k_{\max}\) length sub-frames offset by \(\pm k\), approximating a \(k\)-bin pitch shift.

A common thread across both versions is that the data representation is not merely a frontend choice but part of the learning prior: the log-frequency axis turns pitch transposition into translation, making equivariance operationally enforceable.

## 3. Architecture and transposition-equivariant objective

The 2023 encoder is a shared Siamese network with only \(28.9\)k parameters. It processes each CQT frame independently through LayerNorm, two parallel \(1\)D convolutional layers with kernel size \(3\) and padding \(1\) plus a residual skip connection, then four additional \(1\)D convolutional layers with kernel size \(3\), padding \(1\), LeakyReLU with slope \(0.3\), and dropout rate \(0.2\). A final Toeplitz fully-connected layer followed by softmax maps the representation to a probability distribution over \(d=N\) quantized pitches [2309.02265].

Toeplitz fully-connected layers are introduced to enforce exact transposition equivariance. The weight matrix \(A\in\mathbb{R}^{m\times n}\) satisfies
\[
A_{i,j}=a_{i-j},
\]
so that shifting the input vector results in a corresponding shift of the output. A Toeplitz matrix contains only \(m+n-1\) parameters instead of \(m\times n\), and in the 2023 configuration the final layer therefore contributes only about \(2d-1\) parameters [2309.02265].

The self-supervised objective is formulated on softmax-normalized outputs
\[
y=f(x)\in[0,1]^d,\qquad y^{(k)}=f(x^{(k)}).
\]
The group action \(T_k\) is a discrete shift of the output index, and equivariance requires
\[
f(T_kx)=T_k f(x),
\]
equivalently
\[
y^{(k)}_i=y_{i-k}\quad \forall i.
\]
To enforce this, the method combines three loss terms: an equivariance loss \(L_{\mathrm{equiv}}\) based on a fixed linear projector \(\phi(y)=\sum_{i=0}^{d-1}\alpha^i y_i\) with \(\alpha=2^{1/36}\), a shifted cross-entropy regularizer \(L_{\mathrm{SCE}}\), and an invariance loss \(L_{\mathrm{inv}}\) between pitch-preserving augmentations. The full loss is
\[
L=\lambda_{\mathrm{inv}}L_{\mathrm{inv}}(y,\tilde y)+\lambda_{\mathrm{equiv}}L_{\mathrm{equiv}}(\tilde y,\tilde y^{(k)},k)+\lambda_{\mathrm{SCE}}L_{\mathrm{SCE}}(\tilde y,\tilde y^{(k)},k),
\]
with the weights tuned automatically via gradient-norm balancing [2309.02265].

The 2025 system retains the same high-level structure but scales the backbone. Its single-branch network \(f_\theta:\mathbb{R}^{F'}\to[0,1]^K\) uses seven \(1\)D convolutions along log-frequency, kernel size \(13B=39\), three residual skip-connections, LeakyReLU with slope \(0.3\), dropout \(0.2\), flattening, and a final Toeplitz fully-connected layer. The Toeplitz layer is described as equivalent to a \(1\)D convolution with padding \(K-1\), thereby preserving exact translation equivariance in the final mapping [2508.01488].

In both papers, collapse prevention is a central design constraint. The 2023 description states that no negative sampling is needed because equivariance losses inherently prevent collapse, while the 2025 description uses a full symmetric objective with gradient stops on targets and dynamic loss weighting based on the gradient norms at the last layer [2309.02265] [2508.01488]. A plausible implication is that the method differs from contrastive SSL not only in the absence of labels but also in the absence of explicit negatives, replacing instance discrimination with a structured group-action constraint.

## 4. Training, evaluation, and real-time operation

The 2023 training procedure computes CQT frames of length \(K=297\), crops them to length \(N=K-2k_{\max}\), uses batch size \(256\), and optimizes with Adam at learning rate \(10^{-4}\), default \(\beta\)-parameters, for \(50\) epochs with a cosine-annealing schedule. For each CQT frame, a shift \(k\in\{-16,\dots,16\}\) is sampled along with two random pitch-preserving augmentations [2309.02265].

Evaluation in that paper uses two monophonic pitch-estimation benchmarks: MIR-1K for singing voices and MDB-stem-synth for isolated instrument notes. Metrics are Raw Pitch Accuracy, defined as the percentage of voiced frames with error less than \(0.5\) semitone, and Raw Chroma Accuracy, which ignores octave errors [2309.02265].

For clean signals, the reported RPA results are:

| Model | MIR-1K | MDB |
|---|---:|---:|
| SPICE | 90.6% | 89.1% |
| DDSP-inv | 91.8% | 88.5% |
| CREPE | 97.8% | 96.6% |
| PESTO trained on MIR-1K | 96.1% | 94.6% |
| PESTO trained on MDB | 93.5% | 95.5% |

The same paper reports robustness experiments on MIR-1K with background music. A PESTO model trained on clean data drops from \(94.8\%\) on clean signals to \(50.0\%\) at \(0\) dB vocal-to-background SNR, whereas a PESTO model trained with background-mix augmentation retains approximately \(83\%\) at \(0\) dB and clearly outperforms SPICE in noisy conditions [2309.02265].

The 2025 real-time study evaluates on MIR-1K, MDB-stem-synth, and PTDB. In self-supervised comparison, PESTO with \(130\)k parameters reports \(97.7\%\) RPA on MIR-1K and \(97.0\%\) on MDB-ss, versus SPICE at \(90.6\%\) and \(89.1\%\), and DDSP-inv at \(91.8\%\) and \(88.5\%\) [2508.01488]. Cross-dataset results include PESTO trained on MIR-1K and tested on PTDB at \(87.7\%\) RPA versus CREPE at \(87.1\%\), and PESTO trained on PTDB and tested on MDB-ss at \(96.3\%\) RPA versus PENN at \(51.6\%\) [2508.01488]. Multi-dataset training is reported at \(95.6\%/96.7\%\) RPA on MIR-1K/MDB.

Real-time deployment is enabled by a streamable VQT implementation based on nnAudio but replacing standard convolutions with cached convolutions and a circular memory buffer. The naïve VQT latency is
\[
\frac{w}{2f_s}+\tau,
\]
where \(w\) is the largest filter length and \(\tau\) is the model forward time. The paper then introduces “buffer refilling,” yielding
\[
\delta'=\frac{1}{f_s}\Bigl\lfloor \frac{1}{2}w-m(w-h)\Bigr\rfloor+\tau,\qquad m\in[0,0.5].
\]
With \(m=0.5\), total lag is approximately \(h/(2f_s)+\tau\), and the combined system is reported to have total latency less than \(10\) ms [2508.01488].

The reported Real-Time Factor values are \(0.0354\) on CPU using an i9-12900H, \(0.0032\) on GPU using an RTX A2000, and \(0.0568\) for a YIN baseline on CPU [2508.01488]. This supports the claim that PESTO is suitable for embedded and live applications.

## 5. The multivariate cryptographic scheme named Pesto

A separate research line uses the name **Pesto** for a multivariate public-key construction introduced by Calderini, Caminata, and Villa and analyzed in a later cryptanalytic work [2507.15449]. In that scheme, the public polynomials are obtained by applying a CCZ transformation to a set of quadratic secret polynomials, with the consequence that the public key consists of degree-\(4\) polynomials [2507.15449].

The secret map is defined over a base field \(\mathbb{F}_q\) with positive integers \(n,m,t,s\) satisfying \(t\le \min\{n,m\}\) and \(0<s\le n-t\). The \(n\) input variables are split as
\[
x=(x_1,\dots,x_t)\in \mathbb{F}_q^t,\qquad y=(y_1,\dots,y_{n-t})\in \mathbb{F}_q^{n-t}.
\]
The central map \(F:\mathbb{F}_q^n\to\mathbb{F}_q^m\) is
\[
F(x,y)=(T(x,y),U(x,y)),
\]
where \(T(x,y)=x+Q(y)\) and \(Q(y)=(Q_1(y),\dots,Q_t(y))\), with each \(Q_i\) a random quadratic form in the \(y\)-variables. The second block \(U\) is an Oil-Vinegar system of \(m-t\) quadratics with vinegar variables \(V=\{x_1,\dots,x_t,y_1,\dots,y_s\}\) and oil variables \(O=\{y_{s+1},\dots,y_{n-t}\}\) [2507.15449].

The scheme does not publish \(F\) directly. Instead, it samples affine-linear bijections
\[
A_2:\mathbb{F}_q^n\to\mathbb{F}_q^n,\qquad A_1:\mathbb{F}_q^m\to\mathbb{F}_q^m,
\]
defines a twisted secret map
\[
G(x,y)=\bigl(T_y^{-1}(x),\,U(T_y^{-1}(x),y)\bigr),
\]
with \(T_y^{-1}(x)=x-Q(y)\), and publishes
\[
G_{\mathrm{pub}}=A_1\circ G\circ A_2.
\]
Since the first \(t\) components of \(G\) have degree at most \(2\) and the remaining ones degree at most \(4\), each public component \(P_k(z)=G_{\mathrm{pub},k}(z)\) is a polynomial of total degree at most \(4\) in the public variables \(z_1,\dots,z_n\) [2507.15449].

The intended significance of the design is that a quadratic central map is hidden behind a quartic public representation. This suggests a degree-based obfuscation strategy: the CCZ transform is used not to alter the underlying algebraic structure of the trapdoor, but to disguise it through a higher public degree.

## 6. Cryptanalysis and security implications

The cryptanalytic result argues that the degree-\(4\) public system can be efficiently reduced to a system of quadratic polynomials, which in turn suggests that the CCZ transformation does not offer a significant increase in security [2507.15449]. Two polynomial-time reduction methods are described.

The first method is a Grӧbner-basis-style elimination using a Macaulay matrix. It forms the vector space \(W\) of degree-\(\le 2\) linear combinations of the public polynomials \(P_i(z)-c_i\), then constructs the full Macaulay matrix in degree \(4\), performs Gaussian elimination augmented by the “mutant trick,” and shows that rows of final degree \(3\) or \(4\) are congruent modulo \(W\) to quadratic polynomials. The resulting system has \(m'=O(n^2)\) pure quadratics with the same common zeros as the quartic public equations. The total cost is stated as
\[
O(m^2 n^4+n^{16})
\]
field operations [2507.15449].

The second method is a Higher-Order Linearization Equation attack. It introduces the algebra of symbolic input/output monomials in variables \(z_1,\dots,z_n,w_1,\dots,w_m\), samples roughly \(\binom{n+m+2}{2}\) input guesses \(z^{(1)},\dots,z^{(N)}\), computes the corresponding outputs \(w^{(\ell)}=P(z^{(\ell)})\), stacks the sampled monomial evaluations into a linear system, and extracts from the nullspace at least \(m+t\) linearly independent quadratic relations \(R_{\mathrm{pub}}^{(i)}(z,w)=0\). Substituting \(w=P(z)=c\) then recovers purely quadratic equations in \(z\) alone. The stated complexity is
\[
O\bigl(m n^6+m^3 n^4+m^6\bigr),
\]
which becomes \(O(n^7)\) in the regime \(m\approx n\) [2507.15449].

The cryptanalysis provides numerical rough-order estimates: for \(n=60\), HOLE costs on the order of \(60^7\approx 2\times 10^{12}\) field operations, and for \(n=100\), approximately \(10^{14}\) operations [2507.15449]. It concludes that **Pesto as proposed in [CCV24] is no harder than solving an unbalanced OV system in \(n\) variables and \(m\approx n\) equations**, that **applying a CCZ transform to any central quadratic map admitting a \(t\)-twist cannot yield super-quadratic security**, and that **CCZ offers no meaningful extra layer of security in multivariate-public-key schemes based on low-degree central maps** [2507.15449].

A common misconception in this setting is that quartic public equations necessarily imply quartic hardness. The cryptanalytic result directly contests that interpretation: the public degree is shown not to be a stable indicator of effective algebraic difficulty for this construction.

## 7. Comparative significance of the two usages

The two meanings of Pesto are unrelated in domain and methodology. The pitch-estimation PESTO is a self-supervised neural method for monophonic or single-pitch estimation, built around translation equivariance in log-frequency representations, Siamese training, Toeplitz layers, and real-time streaming constraints [2309.02265] [2508.01488]. The cryptographic Pesto is a multivariate public-key design built from a quadratic central map, an Oil-Vinegar component, and a CCZ transformation intended to yield a quartic public system [2507.15449].

Despite the nominal overlap, the acronymic pitch-estimation literature and the cryptographic scheme employ distinct technical notions of equivariance and transformation. In the audio setting, equivariance is a learning prior: a pitch shift in the input should induce an equal shift in the output distribution. In the cryptographic setting, the relevant transformation is a structural disguise mechanism for public polynomials. The former uses Toeplitz operators to preserve translation symmetry by design; the latter uses affine bijections and CCZ-style composition to obscure an underlying trapdoor map.

The juxtaposition is instructive. In audio MIR, the central claim is that explicit structural constraints can improve sample efficiency, cross-dataset generalization, and real-time deployability without annotated data. In multivariate cryptography, the central claim of the cryptanalysis is nearly the opposite in spirit: a structural transformation intended to increase hardness may preserve enough algebraic regularity to be inverted by polynomial-time linear-algebraic methods. This suggests that “structure” is neither intrinsically beneficial nor intrinsically harmful; its effect depends on whether the application seeks invariance and generalization, as in SSL pitch estimation, or resistance to algebraic reduction, as in public-key design.

Source: https://www.emergentmind.com/topics/pesto