Variable Tempo in Music & Machine Learning

• Variable Tempo is a concept describing non-fixed rates in sequential processes, evident in musical rubato and variable episode durations in AI.
• It spans multiple domains—from bar-level BPM measurement in chamber music to tempo-conditioned synthesis and control in generative systems.
• Methodologies include cumulative timing analysis, probabilistic hidden state models, and tempo-invariant representation learning to enhance expressive control.

Variable tempo denotes the condition that the effective rate of a sequential process is not fixed. In music, it includes local departures from a movement’s mean tempo such as rubato, fermatas, accelerandi, and ritardandi, as well as stable differences in beats per minute across performances and repertoires. In machine learning and generative systems, closely related notions appear as tempo-invariant representations, tempo-conditioned synthesis, wall-clock training intervals, and explicit execution-speed factors. Across these settings, tempo may be treated as an observable, a hidden state, a nuisance factor to be normalized away, or a control variable to be manipulated directly (Sole, 16 Apr 2026, Giorgi et al., 2021, Lee et al., 2023, Jing et al., 4 Jun 2026).

1. Operational definitions across domains

Several precise operationalizations of variable tempo coexist in the recent literature. In empirical performance analysis, tempo is a bar-level quantity derived from cumulative timestamps. In non-stationary reinforcement learning, tempo is the episode-to-episode time allocation of the agent relative to wall-clock drift in the environment. In pitch-based sonification, “Variable Tempo” denotes equal pitch steps with variable inter-onset intervals. In speed-controllable robot policies, tempo is an explicit speed factor used to retime demonstrations and condition action generation (Sole, 16 Apr 2026, Lee et al., 2023, Fan et al., 9 Aug 2025, Jing et al., 4 Jun 2026).

Domain	Tempo variable	Operationalization
Chamber-music timing	$\mathrm{BPM}_i$	$\mathrm{BPM}_i=\frac{n_i\times 60}{\Delta t_i}$
Non-stationary RL	$\Delta t_k$	$\Delta t_k=t_{k+1}-t_k$
Pitch sonification	$\Delta t$	$\Delta t=\Delta y / |f'(x)|$
Vision-language-action	$s$	demonstration retimed to target speed $s>0$

These definitions are not interchangeable, but they are structurally related. Each specifies how temporal spacing varies as a function of either a latent process, an observed signal, or an explicit command. This suggests a useful distinction between variable tempo as measurement of a naturally varying process, variable tempo as inference target in sequential models, variable tempo as representation problem for invariance, and variable tempo as control mechanism in generative or embodied systems.

2. Measurement and inference of time-varying tempo

In music performance analysis, one direct approach is the cumulative lap-timer protocol developed for polyphonic chamber music. A continuous stopwatch is started at the first downbeat and the “lap” button is pressed at each notated barline, producing cumulative timestamps $T_1,\dots,T_m$. Bar duration is then computed as $\Delta t_i=T_i-T_{i-1}$ and instantaneous tempo as $\mathrm{BPM}_i=\frac{n_i\times 60}{\Delta t_i}$0, with a spreadsheet self-check enforcing $\mathrm{BPM}_i=\frac{n_i\times 60}{\Delta t_i}$1 within reaction-time tolerance (Sole, 16 Apr 2026). The method is designed so that a single timing error affects at most two adjacent bars, and it is explicitly motivated by the failure of automated beat-detection software on historical polyphonic chamber recordings. The same study uses the resulting bar-level series to identify rubato, fermatas, accelerandi, and ritardandi, and reports visual analysis through tempographs, histograms with spline-smoothed probability density functions, ridgeline plots, and combination charts.

Probabilistic approaches instead model tempo as a hidden state coupled to rhythmic structure. In the switching state-space model of tempo tracking and rhythm quantization, the discrete variables correspond to note locations in a score and the continuous hidden variables denote tempo. The basic continuous state is $\mathrm{BPM}_i=\frac{n_i\times 60}{\Delta t_i}$2, where $\mathrm{BPM}_i=\frac{n_i\times 60}{\Delta t_i}$3 is the current period and $\mathrm{BPM}_i=\frac{n_i\times 60}{\Delta t_i}$4 the ideal onset time, with Gaussian dynamics

$\mathrm{BPM}_i=\frac{n_i\times 60}{\Delta t_i}$5

and observation model $\mathrm{BPM}_i=\frac{n_i\times 60}{\Delta t_i}$6 (Cemgil et al., 2011). Exact inference is intractable because filtering yields an exponentially growing Gaussian mixture, so the paper develops sequential Monte Carlo and Markov chain Monte Carlo approximations. The reported simulations favor sequential methods, and the particle filter is described as running in real time, with 5–10 particles sufficing on a 800 MHz PC.

A related mixed graphical model formulates rhythmic parsing through discrete rhythm states $\mathrm{BPM}_i=\frac{n_i\times 60}{\Delta t_i}$7, continuous tempo states $\mathrm{BPM}_i=\frac{n_i\times 60}{\Delta t_i}$8, and inter-onset intervals $\mathrm{BPM}_i=\frac{n_i\times 60}{\Delta t_i}$9. Tempo evolves as

$\Delta t_k$0

while observations satisfy

$\Delta t_k$1

The MAP solution is obtained by a Viterbi-style dynamic program in which each $\Delta t_k$2 is maintained exactly as a finite “max” of Gaussian kernels, with a thinning step keeping only kernels that ever dominate (Raphael, 2013).

A third formulation models expressive tempo as piecewise quadratic arcs. The observed instantaneous-tempo sequence is represented by arc segments

$\Delta t_k$3

with Gaussian noise and priors on slope, curvature, and arc duration. MAP inference is then performed by a Viterbi-like dynamic program over candidate breakpoints, yielding a score-agnostic method that supports online analysis and immediate future tempo prediction (Stowell et al., 2013). Taken together, these lines of work treat variable tempo as a latent dynamical process rather than as annotation noise.

3. Tempo invariance and tempo-specific representation learning

A central problem in music information retrieval is that the same rhythmic pattern may appear at substantially different tempi. One solution is to represent rhythmic activity in log-frequency coordinates. In the log-frequency “rhythmogram” approach, per-instrument onset activations are transformed with a Constant-Q Transform over overlapping windows, using 24 bins per octave and periodicities from 1 Hz to 1000 Hz. With $\Delta t_k$4 Hz and $\Delta t_k$5, the tempo-to-bin relation is

$\Delta t_k$6

so doubling the tempo shifts the pattern by exactly one octave, or 24 bins. Convolution with stride 1 across the log-frequency axis, followed by pooling in octave bands, yields quasi-invariance to tempo shifts while relative-phase channels encode inter-channel timing relations independently of absolute CQT phase (Elowsson, 2018).

A more explicit construction is the deterministic time-warping operation for downbeat tracking. Each frame index $\Delta t_k$7 is mapped to a musical-time coordinate through

$\Delta t_k$8

so that a pattern of length $\Delta t_k$9 always spans $\Delta t_k=t_{k+1}-t_k$0 beats regardless of beat period $\Delta t_k=t_{k+1}-t_k$1 (Giorgi et al., 2021). The reported model uses a front-end of three one-dimensional convolutional layers, followed by $\Delta t_k=t_{k+1}-t_k$2 scale-invariant convolutional layers with $\Delta t_k=t_{k+1}-t_k$3 output channels, pattern length $\Delta t_k=t_{k+1}-t_k$4, and $\Delta t_k=t_{k+1}-t_k$5 discrete tempo-scales over $\Delta t_k=t_{k+1}-t_k$6 with $\Delta t_k=t_{k+1}-t_k$7 s. On the synthetic Groove MIDI → FluidSynth benchmark, the tempo-invariant CNN achieves $\Delta t_k=t_{k+1}-t_k$8 on both the train set and the tempo-scaled test set, whereas the conventional CNN achieves $\Delta t_k=t_{k+1}-t_k$9 on training data but drops to approximately $\Delta t$0 on unseen speeds. On real music, the same architecture yields Ballroom $\Delta t$1 versus $\Delta t$2 for the non-invariant baseline, and GTZAN $\Delta t$3 versus $\Delta t$4.

A different strategy does not remove tempo but manipulates it within a learned embedding space. The tempo translation operator $\Delta t$5 is implemented by a small MLP that takes the original 1 728-dimensional embedding $\Delta t$6 and $\Delta t$7:

$\Delta t$8

Training minimizes a cosine-similarity term plus mean-squared error against the embedding of an actually time-stretched spectrogram (McCallum et al., 2024). The translation network is reported to be approximately 40× faster on CPU than the full embedder, and translation-augmented training yields the same improvement as classical spectrogram-time-stretch augmentation for downstream tempo prediction. Retrieval experiments further show that translated embeddings match the audio-stretch baseline in tempo alignment while keeping tag precision within 1–2% of baseline across $\Delta t$9.

These approaches separate two research agendas. Tempo-invariant models attempt to factor tempo out of the representation, whereas translation models retain tempo as a manipulable direction in feature space. A plausible implication is that variable tempo can be treated either as equivalence class structure or as controllable latent geometry, depending on the downstream task.

4. Tempo as an explicit control variable in generation and decision-making

Recent generative systems increasingly model tempo as a first-class conditioning signal. In video-to-audio synthesis, AudioIM separates timbre and tempo through dual encoders. Its tempo encoder receives a short reference clip at 44.1 kHz, uses the “Style Conditioner” with a Residual Vector Quantizer of six distinct codebooks, concatenates the six codebook outputs to form $\Delta t=\Delta y / |f'(x)|$0, and projects to the diffusion hidden dimension $\Delta t=\Delta y / |f'(x)|$1 in MMAudio-L:

$\Delta t=\Delta y / |f'(x)|$2

This embedding is fused with a timbre embedding, incorporated into global and frame-level conditions, and injected through adaptive LayerNorm. No dedicated tempo loss is introduced; tempo control is learned implicitly under the conditional flow-matching objective. On VGGSound, the paper reports KL_PANNs improvement from 1.95 to 1.85, KL_PaSST from 1.72 to 1.63, and Style-Similarity MOS from $\Delta t=\Delta y / |f'(x)|$3 to $\Delta t=\Delta y / |f'(x)|$4 (Zhao et al., 5 Jun 2026).

In music-to-3D-dance generation, TempoMoE organizes model capacity directly by tempo range. The method discretizes 60–200 BPM into $\Delta t=\Delta y / |f'(x)|$5 overlapping tempo bands centered at $\Delta t=\Delta y / |f'(x)|$6 BPM. Each tempo band contains three beat-scale experts operating at $\Delta t=\Delta y / |f'(x)|$7-beat, $\Delta t=\Delta y / |f'(x)|$8-beat, and $\Delta t=\Delta y / |f'(x)|$9-beat resolutions, with kernel size

$s$0

Routing is hierarchical: hard TopK tempo-group selection with $s$1 in practice, followed by a softmax over the three beat-scale experts within each active group (Lyu et al., 21 Dec 2025). Reported results show best $s$2 of 25.13 versus next-best 28.16, highest $s$3 and $s$4, and highest Beat Alignment Score of 0.2446 versus 0.2423. Sample-level router statistics indicate that slow inputs around 64 BPM emphasize low-BPM groups and whole-beat experts, whereas fast inputs around 185 BPM emphasize high-BPM groups and quarter-beat experts.

In robotic manipulation, TempoVLA introduces bidirectional speed control for a single vision-language-action policy. Its Variable-Speed Trajectory Augmentation re-times a trajectory to target speed $s$5 by accumulating motion over blocks of $s$6 actions,

$s$7

and re-splitting that motion into $s$8 equal-magnitude substeps, preserving integrated motion exactly (Jing et al., 4 Jun 2026). The conditioned policy can be realized through a textual prefix, Speed-Modulated RMSNorm, or soft prompt anchors; all three schemes are reported to achieve essentially identical success rates of approximately 96.8% on average, and the textual prefix is adopted by default. On LIBERO tasks, realized Model Ratio closely tracks the requested speed in the trained range $s$9, while task success remains approximately 97% across this range. In real-world Franka experiments, default 1× success improves from 80.0% to 88.0%, and dynamic scheduling by a vision-LLM achieves 96% success with average realized speedup of 1.21×.

An analogous control problem appears in non-stationary reinforcement learning, where agent tempo is the time-allocation sequence $s>0$0 and environment tempo is the drift of $s>0$1 and $s>0$2 over wall-clock time. ProST minimizes an upper bound on dynamic regret,

$s>0$3

yielding a closed-form suboptimal $s>0$4 that trades off training time against environment change rate (Lee et al., 2023). The paper gives three explicit regimes for $s>0$5 depending on the drift exponent $s>0$6 and reports sublinear dynamic regret in tabular analysis together with superior online return in high-dimensional non-stationary environments relative to fixed-interval baselines.

5. Non-uniform rescaling, retiming, and perceptual encoding

Variable tempo is not always modeled as a global scalar. In Carnatic music, the relevant structure is explicitly non-uniform. The signal is decomposed into Constant-Pitch segments, transients or gamakas, and silence, using non-overlapping frames of length $s>0$7 ms and rules based on pitch range and slope (Viraraghavan et al., 2017). The central empirical observation is that when overall tempo changes, the duration of transients does not change proportionally: across nine concert renditions, CP-notes slow by factors ranging from approximately 6.2 to 12.3, transients only approximately 1.9 to 3.5×, and silences 2.3 to 5.4×. The resulting tempo-change model applies $s>0$8, $s>0$9, and $T_1,\dots,T_m$0, with distinct index mappings for CP-notes, transients, and silences. In listening tests on 18 listeners, the proposed algorithm was preferred in 84% of cases, rising to 90% among experts.

Video re-timing also benefits from explicitly variable speed. Video-ReTime first trains a dense per-frame slowness predictor through self-supervision, generating training clips by random non-uniform speedups with frame-skip factors in $T_1,\dots,T_m$1 and a first-order Markov chain over skip changes. At inference, the model predicts per-frame slowness and an optimization chooses output skip lengths $T_1,\dots,T_m$2 to match both the predicted speed profile and the target duration. The total objective is

$T_1,\dots,T_m$3

with penalties for violating the learned speediness profile, the length constraint, the no-slow-down condition $T_1,\dots,T_m$4, and smoothness of successive skip changes (Jenni et al., 2022). On Kinetics-V, dense speed-prediction accuracy is approximately 85.6% versus 62.9% for a sliding-window SpeedNet baseline, and inference is approximately 12× faster. In a user study comparing variable-tempo editing with uniform speed-up, 64.5% of votes and 84.8% majority preferred the variable-tempo edit as more natural.

In pitch-based sonification, “Variable Tempo” is itself the sampling rule. Instead of uniform $T_1,\dots,T_m$5-spacing, samples are taken at uniform $T_1,\dots,T_m$6-spacing so that successive notes have constant pitch interval $T_1,\dots,T_m$7 while their timing varies with local slope. For a general function $T_1,\dots,T_m$8 linearly mapped to time, the inter-onset interval is

$T_1,\dots,T_m$9

Steep regions therefore produce faster note sequences and flat regions slower ones (Fan et al., 9 Aug 2025). Across two psychoacoustic experiments, this method produces markedly better derivative perception than Continuous or Variable Pitch Interval baselines. In the acceleration-discrimination task, raw JND averages are approximately 0.463 for Continuous, approximately 0.480 for Variable Pitch Interval, and 0.0344 for Variable Tempo, described as over 13 times finer. Model-based JND estimates are 0.27, 0.24, and 0.01 respectively. Participants also report higher confidence, lower mental effort, and stronger preference for Variable Tempo.

These cases share a common principle: temporal remapping is not uniformly distributed over the signal. Instead, tempo variation is concentrated where the structure permits it or where perception benefits most from it.

6. Historical distributions, stability, and methodological debates

A common assumption in empirical performance studies is that tempo changes historically along a single linear trajectory. Recent corpus analysis of Beethoven’s piano and cello sonatas argues against that interpretation. Using bar-level BPM data from over one hundred recordings spanning 1930–2012 and applying $\Delta t_i=T_i-T_{i-1}$0-means clustering with $\Delta t_i=T_i-T_{i-1}$1, the study reports that every movement supports at least two, and usually three, discrete tempo traditions—slow, mid-range, and fast—with the mid-range cluster typically comprising 55–70% of recordings (Sole, 17 Apr 2026). In all but one case, intra-cluster regression slopes are negligible, with $\Delta t_i=T_i-T_{i-1}$2; the single significant exception is the mid cluster of Op. 102 No. 1 Allegro con brio, where $\Delta t_i=T_i-T_{i-1}$3 BPM yr$\Delta t_i=T_i-T_{i-1}$4, $\Delta t_i=T_i-T_{i-1}$5, $\Delta t_i=T_i-T_{i-1}$6, corresponding to an approximate 3.2 BPM deceleration across the study period.

The same work reports that a slow cluster is absent from fast-character movements such as the Op. 5 Rondos and the Op. 69 Scherzo, and finds no correlation between cluster membership and performers’ generational, national, or pedagogical backgrounds. It therefore proposes an ecological model in which the aggregate mean tempo at time $\Delta t_i=T_i-T_{i-1}$7 is

$\Delta t_i=T_i-T_{i-1}$8

so historical change reflects shifts in the prevalence of stable traditions rather than uniform evolution of a single tradition.

This reframing bears directly on the treatment of variable tempo as data rather than noise. The manual bar-by-bar protocol explicitly argues that manual annotation is not a methodological retreat but a principled response to intrinsic limitations of computational tools on polyphonic historical recordings (Sole, 16 Apr 2026). The clustering study likewise challenges the misconception that corpus-level tempo history is necessarily unidirectional. A plausible implication is that variable tempo is often structured at multiple levels simultaneously: within-bar or within-phrase expressive modulation, movement-level mean tempo, and long-run coexistence of distinct interpretive regimes.