ScaleMaster Datasets: TSM and Monocular SLAM

• ScaleMaster Dataset is a collection of two domain-specific benchmarks, addressing audio time-scale modification with perceptual quality labels and monocular SLAM with scale consistency challenges.
• The audio TSM dataset includes over 5,500 processed clips with detailed subjective ratings to benchmark artifacts like phasiness, transient smearing, and other distortion effects.
• The monocular SLAM benchmark offers indoor sequences with ARKit and LiDAR references to evaluate trajectory errors, map consistency, and scale ambiguity over varied building-scale trajectories.

Searching arXiv for the ScaleMaster papers to ground the article in the cited records. ScaleMaster is the name of two distinct research datasets introduced in different domains. In audio signal processing, it denotes a 2020 corpus for time-scale modification (TSM) containing processed audio clips with subjective quality labels, created to support objective prediction of perceived TSM quality (Roberts et al., 2020). In deep visual SLAM, it denotes a 2026 benchmark for evaluating scale consistency in monocular systems operating in large-scale indoor environments, with trajectory and map-based evaluation against ARKit and LiDAR references (Ju et al., 20 Feb 2026). The shared name reflects a common concern with “scale,” but the term has domain-specific meanings: playback rate and duration transformation in TSM, and metric consistency of trajectories and reconstructions in monocular SLAM.

1. Disambiguation and scope

The two ScaleMaster datasets address different technical problems and should not be conflated. The earlier dataset concerns perceptual audio quality after time stretching or compression without pitch or timbre change. The later dataset concerns failure modes of monocular SLAM under long trajectories, vertical motion, repetitive views, and low-texture regions.

Dataset	Domain	Core purpose
ScaleMaster (Roberts et al., 2020)	Time-scale modification	Train and validate objective measures of subjective TSM quality
ScaleMaster (Ju et al., 20 Feb 2026)	Deep monocular visual SLAM	Evaluate intra-session scale inconsistency and inter-session scale ambiguity

In the TSM setting, “scale” is formalized through the duration ratio $\alpha$ and playback-speed ratio $\beta$, with $\beta = \frac{S_a}{S_s} = \frac{1}{\alpha}$ and playback rate $\rho = \beta$. In the monocular SLAM setting, scale is not directly observable from monocular imagery and must be inferred up to a similarity transform; the benchmark therefore evaluates performance after a single global Sim(3) alignment.

2. ScaleMaster in time-scale modification

The TSM ScaleMaster dataset was created to fill what the paper describes as a long-standing gap: the absence of a shared, content-diverse, subjectively labeled corpus for developing and benchmarking objective TSM quality measures (Roberts et al., 2020). The motivating premise is that many TSM algorithms exist, but effective objective measures of perceived quality do not, and prior subjective tests were difficult to compare because they used different sources, methods, and time scales. The dataset therefore targets canonical TSM artifacts, including phasiness and reverberation, musical or metallic noise, transient smearing, and the “buzzy” quality caused by transient skipping or duplication.

Its composition is explicitly split into training and testing components. The training set contains 88 source files processed by six TSM methods at 10 time-scale ratios, yielding 5,280 processed files. The testing set contains 20 source files processed by three additional methods at four time scales, yielding 240 processed files. The source material spans 34 musical files, 37 solo instrument files, and 37 voice files, plus an evaluation subset. Content includes speech by male, female, and child speakers, singing, harmonic monophonic instruments such as winds and strings, percussive instruments, piano, synthetic basses, and multi-source music across classical, rock, jazz, and electronic genres; sound effects also appear within the musical, solo, and evaluation groupings.

All material is mono, formed by left-right averaging to avoid multi-channel processing pitfalls. Audio is provided at 44.1 kHz, 16-bit PCM WAV, with amplitude normalized to $\pm 1$ prior to TSM. The reported sound-pressure levels span roughly 56.6–86.9 dB, with mean 73.37 dB and standard deviation 6.75 dB. Clip durations were kept short, with mean 3.7 s and standard deviation 1.6 s, so that scaled excerpts remained manageable.

The six training methods were Phase Vocoder (PV), Identity Phase Locking PV (IPL), Waveform Similarity Overlap-Add (WSOLA), Fuzzy Epoch-Synchronous OLA (FESOLA), Harmonic-Percussive Separation TSM (HPTSM), and Multi-component Time-Varying Sinusoidal decomposition (uTVS). The three testing methods were Elastique, FuzzyPV, and NMFTSM. The training set used 10 exact playback-speed ratios $\beta$: 0.3838, 0.4427, 0.5383, 0.6524, 0.7821, 0.8258, 0.9961, 1.381, 1.667, and 1.924. Testing used four ratios per method selected randomly from four bands spanning $0.25 \le \beta \le 2$, although the exact testing values are not enumerated in the paper.

The parameterization of the author implementations is specified. PV and IPL use Hann windows, frame length 2,048 samples and synthesis hop 512 samples. FESOLA uses frame length 1,024. WSOLA uses frame length 1,024, synthesis hop 512, and tolerance 512. HPTSM uses IPL parameters on the harmonic branch and WSOLA with frame 256 and hop 64 on the percussive branch. uTVS uses $6\times$ oversampling and a Mel filterbank with 88 filters. Elastique, FuzzyPV, and NMFTSM were run with provided implementations and default settings. All processed files were amplitude normalized post-processing. A notable caveat is that a coding bug affected uTVS near $\beta \approx 1$ for some files; the outputs were retained for artifact diversity and later fixed for the evaluation subset.

3. Subjective labeling, normalization, and reliability in the TSM dataset

The TSM dataset’s labels are based on reference–processed pairwise presentation with free replay, subject to the requirement that each item in the pair be heard at least once (Roberts et al., 2020). Ratings were provided on a continuous control mapped to a 1–5 quality scale anchored by Poor and Excellent and reported as mean opinion scores (MOS). In-lab collection used a MATLAB GUI, with sessions initially containing about 200 files and lasting about 18 minutes. Participants received explanations of TSM purpose and common artifacts, heard audio examples, and completed a 33-file practice set. Remote collection used the Web Audio Evaluation Tool (WAET); session length was reduced from 100 to 60 pairs in response to feedback. Remote sessions collected name, age, transducer, experience level, and any known hearing issues, and participants were reminded to use headphones in a quiet space.

The dataset contains 42,529 file ratings from 263 participants across 633 sessions, including 10,354 in-lab ratings. The age range was 16–66 years, with median 30, and 52.36% of ratings were contributed by expert listeners. For anchoring the scale extremes, 12 files had MOS truncated to 1 and 28 files to 5.

The paper normalizes opinion scores per ITU-R BS.1284 to remove rater and session bias. For a score $x_i$, the transformation is

$\beta$0

where $\beta$1 and $\beta$2 are the participant mean and standard deviation within session $\beta$3, and $\beta$4 and $\beta$5 are the session mean and standard deviation across participants on the same file set. MOS were limited to the range 1–5. Outlier sessions were removed using robust thresholds based on scaled median absolute deviation applied to RMSE and Pearson correlation relative to file-level MOS; 45 sessions, corresponding to 2,102 ratings or 4.94%, were removed.

Reliability improved after outlier removal and normalization. Mean per-session RMSE improved from 0.771 to 0.682, and mean Pearson correlation improved from 0.791 to 0.799. Interrater reliability, estimated using the Ill-Structured Measurement Designs coefficient $\beta$6 rather than ICC, increased from 0.871 before normalization to 0.909 after normalization. File-level MOS standard deviation decreased from 0.802 to 0.718, and convergence analysis indicated that approximately seven ratings per file yielded stable MOS.

The listener-factor analysis is notable because it directly addresses common concerns about crowdsourced perceptual evaluation. Two one-sided tests showed equivalence between expert and non-expert listeners on both PCC and RMSE at $\beta$7 intervals of the expert means, with maximum p-values 0.0498 for RMSE and $\beta$8 for PCC, and small effect sizes of approximately 0.13 and 0.11. Hearing issues produced equivalent PCC but not equivalent RMSE at $\beta$9; however, a t-test could not reject equal means for RMSE, and equivalence could be claimed at $\beta = \frac{S_a}{S_s} = \frac{1}{\alpha}$0. Laboratory and remote collection were equivalent on PCC, while RMSE equivalence required a wider interval of $\beta = \frac{S_a}{S_s} = \frac{1}{\alpha}$1; a t-test again could not reject equal means. Age had negligible impact on rating quality, with correlations 0.108 to RMSE and $\beta = \frac{S_a}{S_s} = \frac{1}{\alpha}$2 to PCC. These findings support the use of controlled browser-based collection for this task.

4. Findings, objective-measure benchmarking, and experimental use in TSM

The dataset was designed not merely as a repository of processed audio, but as a benchmark for method- and content-dependent quality behavior (Roberts et al., 2020). Subjective analysis showed that perceived quality improves as $\beta = \frac{S_a}{S_s} = \frac{1}{\alpha}$3 for all methods. Slowing, corresponding to $\beta = \frac{S_a}{S_s} = \frac{1}{\alpha}$4, was generally harder than speeding, corresponding to $\beta = \frac{S_a}{S_s} = \frac{1}{\alpha}$5; when inverse pairs such as 0.5 and 2 were compared, the slower condition yielded lower MOS. Artifact preference depended on content and scale. For $\beta = \frac{S_a}{S_s} = \frac{1}{\alpha}$6, PV was preferred over WSOLA, suggesting that listeners found phasiness and smearing less objectionable than transient doubling. For $\beta = \frac{S_a}{S_s} = \frac{1}{\alpha}$7, WSOLA was preferred, suggesting that transient skipping was less objectionable than PV artifacts in that regime. IPL sometimes trailed PV at large reductions because of metallic artifacts. Male voices were harder than female or child voices, consistent with lower frequency-resolution challenges; synthesizer bass and noisy percussion were difficult, while clean harmonic monophony received the highest ratings.

The paper also uses the dataset to test whether published objective measures predict subjective quality. TSM-specific measures included Signal-to-Error Ratio (SER) and synthesis consistency $\beta = \frac{S_a}{S_s} = \frac{1}{\alpha}$8. Their average absolute correlations with MOS across $\beta = \frac{S_a}{S_s} = \frac{1}{\alpha}$9 and $\rho = \beta$0 were low: 0.3707 for SER and 0.1574 for $\rho = \beta$1. The reported limitations are lack of phase awareness for SER and poor indication of phasiness for $\rho = \beta$2. PEAQ, originally defined for codec distortions on time-aligned signals, was adapted by interpolating reference magnitude spectra to the duration of the test signal, because direct application is invalid under TSM-induced time misalignment. Even with this adaptation, PEAQ achieved correlations similar to SER and $\rho = \beta$3, indicating that unmodified codec-oriented perceptual measures are poorly matched to TSM-specific distortions such as phase incoherence, transient relocation, and envelope changes.

A retraining experiment was then performed on the basic PEAQ neural network and feature set. The protocol reserved 10% of the training data for validation, ran 100 random seeds, and selected the epoch minimizing an overall distance metric that jointly favors high PCC and low RMSE across train, validation, and test splits. The best network achieved average RMSE 0.668, average PCC 0.719, and joint distance metric $\rho = \beta$4. The paper states that these values approach the distributions of subjective sessions, placing at the 11th and 17th percentiles respectively, and substantially outperform SER and $\rho = \beta$5.

The intended usage protocol reflects these design choices. Training is recommended on the 5,280 processed clips from the six training methods, with a validation subset stratified across $\rho = \beta$6 and content categories. Evaluation is recommended on the 240 processed clips produced by the three unseen testing methods, so that generalization to new algorithms and previously unseen $\rho = \beta$7 values can be assessed. RMSE and Pearson correlation against MOS are identified as the primary metrics, with reporting across content classes and $\rho = \beta$8 bands to expose failure modes. The paper also recommends keeping clips from the same source file segregated across train, validation, and test when content-sensitive feature learning is used, preserving the method split, normalizing ratings per BS.1284, and removing outlier sessions using scaled MAD thresholds. The dataset is available through IEEE Dataport at http://ieee-dataport.org/1987 under CC BY 4.0.

5. ScaleMaster in deep monocular visual SLAM

The 2026 ScaleMaster benchmark addresses a different problem: the evaluation of scale consistency in deep monocular visual SLAM under realistic indoor conditions that are underrepresented in existing datasets (Ju et al., 20 Feb 2026). The paper argues that benchmarks such as TUM-RGBD, EuRoC, 7-Scenes, and ARKitScenes permit high absolute accuracy on short or structurally simple sequences, but do not expose the failure modes that arise in building-scale trajectories with vertical motion, repetitive structures, and low-texture regions. The benchmark therefore focuses on two challenges. The first is intra-session scale inconsistency, in which monocular odometry drifts and Sim(3) pose-graph optimization may yield scale explosions or warped maps even when globally aligned trajectories appear reasonable. The second is inter-session scale ambiguity, in which maps of the same environment reconstructed in separate sessions converge to incompatible scales and cannot be fused reliably.

The dataset contains 25 monocular RGB sequences collected in large, complex indoor environments, with seven sequences augmented by high-fidelity LiDAR maps and 18 providing trajectory-only reference. The environments include large halls, library floors with multi-floor descents and ascents, long corridors, offices with repetitive shelving or desk patterns, underground parking, lobbies, lounges, hotel rooms, labs, basements, stairwells, and a station escalator. The paper reports a minimum trajectory length of 3.51 m for Library_08, median length 109.89 m for LargeHall_03, and maximum length 884.12 m for LargeHall_01. The set spans short, medium, and long trajectories and deliberately includes building-scale loops.

Several sequences are highlighted for their stress characteristics. Library_01 traverses from the fifth floor to the third, Stairs_01 from the second floor to the sixth, Stairs_02 repeats upward and downward motion, and Station_01 contains an escalator. LargeHall_01, Parking_01, and Library_01 exemplify long trajectories and loops. Repetitive views arise from library shelving, offices, and stairwell patterns, while low-texture and low-light conditions occur in large open atria, evening hall sequences, and several library or hall scenes.

Capture uses monocular RGB from an iPhone 14 Pro at 1920×1440 resolution, with frame counts and durations indicating approximately 30 Hz. Seven sequences additionally include dense 3D point clouds from a Livox HAP LiDAR. An Orbbec Gemini 335L is also part of the rig, and the iPhone provides ARKit odometry used as the baseline trajectory for constructing reference maps. The paper emphasizes that the custom handheld rig ensures precise temporal synchronization between visual frames and LiDAR scans. It does not enumerate camera intrinsics or distortion models in the text and directs readers to the dataset website for calibration assets if provided.

Ground truth is built in two layers. ARKit provides reference trajectories described as having centimeter-level accuracy in large indoor spaces, sufficient to expose scale failures. LiDAR point clouds projected using ARKit poses form dense metric reference maps for the seven LiDAR-equipped sequences. The representation is point-cloud based; no meshes are reported. This setup supports a benchmark that compares not only trajectories but also reconstructed geometry.

6. Evaluation protocol, baseline systems, and observed SLAM failure modes

The SLAM ScaleMaster benchmark is evaluation-oriented and does not define train, validation, and test splits (Ju et al., 20 Feb 2026). Its protocol first runs a monocular SLAM system on the input RGB frames and exports camera poses. Estimated poses are then time-synchronized with ARKit trajectories and globally aligned through a single Sim(3) transform $\rho = \beta$9 obtained by the Umeyama solution to

$\pm 1$0

with $\pm 1$1, $\pm 1$2, and $\pm 1$3. Absolute Trajectory Error is then computed after alignment as

$\pm 1$4

following evo. The same single Sim(3) is inherited by the SLAM-generated map for map-to-map comparison against the LiDAR reference. No segment-wise re-alignment is performed, and no additional optimization beyond each method’s native loop closure or pose-graph optimization is added by the benchmark.

Map quality is assessed with Chamfer distance and Drop Rate. For reconstructed point set $\pm 1$5 and reference point set $\pm 1$6, the benchmark uses

$\pm 1$7

and

$\pm 1$8

The paper illustrates $\pm 1$9 m and $\beta$0 m, using the smaller threshold to expose severe local failures and the larger threshold to expose large-scale distortions. Relative Pose Error is defined in standard form but is not the focus of the quantitative tables. The benchmark also notes explicitly that it does not define a dedicated scalar intra-session drift index or inter-session ambiguity scalar; these phenomena are instead revealed through ATE, map-to-map metrics, and qualitative analysis.

The baseline systems are DROID-SLAM, MASt3R-SLAM, MASt3R-SLAM in calibrated mode, and VGGT-SLAM, all run from official repositories with official pre-trained models and without fine-tuning on ScaleMaster. Hardware is reported as an AMD Ryzen 9 9900X CPU and NVIDIA RTX 5090 GPU. On ARKitScenes, which serves as a room-scale comparison benchmark, the average ATE values reported are 0.32 m for DROID-SLAM, 0.24 m for VGGT-SLAM, 0.09 m for MASt3R-SLAM, and 0.05 m for MASt3R-SLAM*. On ScaleMaster, the same systems exhibit large failures on longer or more complex trajectories. On LargeHall_01, which is 884 m long, ATE rises to 89.35 m for DROID-SLAM, 80.54 m for MASt3R-SLAM, and 91.62 m for MASt3R-SLAM*. Parking_01 yields 10.21 m, 32.37 m, and 26.13 m respectively. Stairs_01 yields 20.20 m, 4.60 m, and 2.30 m. In a repetitive office sequence, Office_01, the corresponding numbers are 5.61 m, 8.03 m, and 0.65 m. By contrast, on the short in-place-rotation sequence Library_06, the best ATE is approximately 0.04–0.05 m.

The map-based results show why trajectory-only evaluation is insufficient. For MASt3R-SLAM on Library_06, a successful case, Chamfer distance is approximately 0.10 m at $\beta$1 m and Drop Rate is 0.0%, which supports the interpretation that low ATE corresponds to a geometrically consistent map. For Library_07, however, the benchmark reports a catastrophic failure despite low ATE: Drop Rate is 89.1% at $\beta$2 m and Chamfer distance is approximately 9.99 m at $\beta$3 m. The paper uses this contrast to demonstrate that global Sim(3)-aligned trajectories can appear acceptable while the underlying dense map has undergone scale collapse or severe warping.

The qualitative failure analysis identifies several recurrent modes. Station_01 exhibits short-term scale drift caused by repetitive escalator patterns that degrade short-baseline data association and create overlapping map layers. Parking_01 illustrates long-term Sim(3) pose-graph optimization failure: loop closure is detected, but accumulated scale drift leaves the optimizer in a poor local minimum, so poses may align globally while the map remains warped. Inter-session scale ambiguity is demonstrated by splitting a single long sequence into three independent runs, which produce maps at mutually inconsistent scales and cannot be merged coherently without external scale constraints. The paper attributes these behaviors to weak monocular scale priors, brittleness of pose-graph scale regularization, degenerate geometry in repetitive or low-texture environments, and the ability of trajectory-only Sim(3) optimization to conceal map-scale defects when no joint Sim(3) bundle adjustment is applied over structure and poses.

7. Methodological significance, limitations, and common misconceptions

Taken together, the two ScaleMaster datasets establish benchmarks in which “scale” is the central failure variable, but they operationalize that variable very differently. In the TSM dataset, scale refers to controlled duration and playback transformations with content-dependent perceptual artifacts. In the SLAM benchmark, scale refers to consistency of the latent metric geometry reconstructed from monocular imagery. A plausible implication is that the shared naming reflects a benchmark philosophy rather than a shared application domain: both datasets were built to expose weaknesses that remain hidden when evaluation relies on overly narrow metrics (Roberts et al., 2020, Ju et al., 20 Feb 2026).

Each dataset also corrects a domain-specific misconception. In TSM, the misconception is that classical signal-difference measures or unmodified codec-oriented perceptual measures are adequate proxies for perceived quality. The reported low correlations of SER, $\beta$4, and adapted PEAQ with MOS contradict that assumption and motivate TSM-specific supervised predictors. In monocular SLAM, the misconception is that low globally aligned ATE is sufficient to establish robust reconstruction quality. The contrast between acceptable trajectory error and catastrophic Chamfer or Drop Rate values shows that mapping quality must be evaluated directly.

The limitations of the two resources are explicit. The TSM dataset is mono, 44.1 kHz, 16-bit, and relatively short in duration; multi-channel and long-form audio are not represented. The uTVS implementation bug near $\beta$5 affected some files, although the outputs were retained for artifact diversity and fixed for the later evaluation subset. Ratings are more variable at mid-range quality, and stable MOS requires roughly seven responses per file. For the SLAM benchmark, only seven of the 25 sequences provide LiDAR reference maps, the paper does not specify the license in text, and it does not define closed-form scalar metrics for intra-session scale drift or inter-session ambiguity. Instead, it relies on ATE, Chamfer distance, Drop Rate, and qualitative overlays to expose those effects.

The recommended usage patterns differ accordingly. The TSM ScaleMaster dataset is intended for supervised learning and controlled generalization tests, with a clear training/testing split in which the testing methods are unseen during training. The SLAM ScaleMaster dataset is intended primarily for evaluation of systems run as-is on challenging monocular sequences, with emphasis on single global Sim(3) alignment, inherited map transforms, and per-sequence reporting. The TSM dataset is distributed through IEEE Dataport under CC BY 4.0, whereas the SLAM dataset is released through https://scalemaster-dataset.github.io/, with licensing to be checked on the website.