Perov-5 Dataset: Materials & Traffic Benchmarks

• Perov-5 is a dual-domain dataset, representing both 18,928 perovskite unit cells for polymorphic structure prediction and refined car-following trajectories for traffic analysis.
• In materials informatics, the dataset employs polymorph-aware splitting and evaluation metrics to realistically assess generative models for crystal structure prediction.
• In autonomous driving research, the dataset provides high-quality H-A and H-H car-following segments enhanced via filtering, Kalman filtering, and wavelet denoising techniques.

Searching arXiv for papers related to "Perov-5 Dataset" to ground the article in current literature. Perov-5 is a context-dependent dataset designation used in two distinct research areas. In materials informatics, perov-5 denotes a benchmark of perovskite crystal structures used for crystal structure prediction and benchmark design; in autonomous-driving research, Perov-5 also denotes a processed car-following dataset extracted from the Lyft Level-5 Open Dataset. For clarity, the Editor’s terms “materials Perov-5” and “traffic Perov-5” are used below. The materials benchmark is centered on polymorph-aware evaluation of generative models, whereas the traffic dataset is centered on extracting and denoising human-following trajectories for mixed-traffic analysis (Martirossyan et al., 15 Sep 2025, Li et al., 2023).

1. Terminology and domain scope

The same name has been applied to two unrelated datasets. This suggests a naming ambiguity in recent literature and makes domain context essential when interpreting references to “Perov-5.”

Usage	Domain	Core object
materials Perov-5	crystal structure prediction	perovskite unit cells
traffic Perov-5	car-following analysis	CF trajectory segments

In the materials literature, perov-5 is described as a set of 18,928 perovskite unit cells, each containing exactly five atoms, and originally curated by Castelli et al. from the Materials Project (Martirossyan et al., 15 Sep 2025). In the traffic literature, Perov-5 is a processed car-following dataset derived from the open Lyft Level-5 data and organized into H-A and H-H trajectory segments (Li et al., 2023).

A common misconception is to treat “Perov-5” as an unambiguous benchmark name. The literature summarized here does not support that assumption. Instead, the same label refers to two technically unrelated resources with different data models, evaluation protocols, and scientific purposes.

2. Materials Perov-5: composition, structure count, and polymorphism

The materials Perov-5 benchmark contains 18,928 perovskite unit cells and 9,646 distinct reduced compositions. Of these compositions, 9,282 appear exactly twice as polymorph pairs, while 364 compositions appear only once. The reported polymorphism load is roughly 98% of compositions are polymorphic, making perov-5 an extremely polymorph-rich test set (Martirossyan et al., 15 Sep 2025).

These counts are central to how the benchmark behaves. Because most compositions have more than one structurally distinct entry, the benchmark is not merely a composition-to-structure lookup problem. It is, instead, a one-to-many prediction setting in which multiple legitimate structural realizations may correspond to the same stoichiometry.

The paper illustrates this with explicit polymorph clusters. CaCdSO$_2$ is reported to have two distorted perovskite prototypes, with S and O swapped/enriched. HfNbN$_3$ is reported to have two ABX$_3$ variants with Hf and Nb exchanged on the A/B crystallographic sites. These examples clarify why a benchmark based only on composition identity can obscure structurally meaningful diversity.

A plausible implication is that perov-5 is most informative when used to evaluate structural coverage rather than only single-reference recovery. That interpretation is consistent with the later introduction of polymorph-aware splitting and metrics.

3. Random splitting failure and polymorph-aware resplitting

The benchmark critique focuses on the original split attributed to Xie et al.’s random 60–20–20% division of the 18,928 structures into train, validation, and test subsets. In that split, polymorph pairs are frequently separated across subsets. The reported composition overlap is substantial: 2,265 validation structures share compositions with the training set, and 2,214 test structures share compositions with the training set. Yet among those overlapped compositions, only 94 validation structures and 107 test structures “match” within the standard StructureMatcher tolerances, despite sharing the same stoichiometry (Martirossyan et al., 15 Sep 2025).

The stated consequence is that a generative model is penalized for failing to reproduce a specific polymorph that it was never trained on, which artificially depresses the reported match rate. This is the core methodological objection to random splitting in a highly polymorphic dataset.

To address this, the paper proposes “perov-5-polymorph-split”, which treats each reduced composition and all of its polymorphs as an indivisible group. Formally, with $C=\{c_1,\ldots,c_n\}$ the set of 9,646 unique compositions, and $$P(c)=\{s\in\text{perov-5}:\mathrm{composition}(s)=c\}$$ the polymorph set of a composition, the split partitions $$C=C_{\text{train}}\cup C_{\text{val}}\cup C_{\text{test}}$$ so that

$$|C_{\text{train}}| \approx 0.60|C|,\quad |C_{\text{val}}| \approx 0.20|C|,\quad |C_{\text{test}}| \approx 0.20|C|.$$

The corresponding structure subsets are then defined as

$$S_{\text{train}}=\bigcup_{c\in C_{\text{train}}} P(c),\quad S_{\text{val}}=\bigcup_{c\in C_{\text{val}}} P(c),\quad S_{\text{test}}=\bigcup_{c\in C_{\text{test}}} P(c).$$

The approximate numerical outcome is 5,788 train compositions, 1,929 validation compositions, and 1,929 test compositions, corresponding to about 11,576 train structures, 3,857 validation structures, and 3,857 test structures. By construction, no polymorph pair is ever split across subsets. Models trained on $S_{\text{train}}$ are therefore evaluated on entirely new compositions, which may themselves have multiple polymorphs.

4. Polymorphism-aware evaluation metrics

The benchmark critique also targets the standard one-to-one “match rate”, defined as the fraction of generated structures matching a single reference structure under StructureMatcher tolerances. The paper argues that this metric is blind to polymorphic multiplicity and therefore mis-specified for one-to-many structure generation (Martirossyan et al., 15 Sep 2025).

Two corrected metrics are introduced. The first is METRe (Match-Everyone-To-Reference) rate. If $N_{\text{test}}=|S_{\text{test}}|$, and for each reference structure $_3$0, $_3$1 is the minimum StructureMatcher RMSE among all generated structures of the same composition, then a reference is counted as matched if $_3$2, and

$_3$3

The second is cRMSE (corrected RMSE), defined as

$_3$4

Here, $_3$5 is the mean RMSE over the matched subset, and non-matched references are penalized by assigning them an RMSE = stol.

The stated rationale is twofold. METRe allows each test reference, including each polymorph, to find some generated structure that matches it, thereby rewarding coverage of polymorphic diversity. cRMSE restores a continuous error penalty, combining coverage and positional error so that models cannot improve the metric by matching only a small set of easy references.

5. Benchmark behavior and scientific implications

Under the paper’s reported evaluation, the benchmark behavior changes materially when both the split and the metric are made polymorph-aware. The random-split baseline on perov-5 yields METRe ≈ 58%, mean RMSE ≈ 0.09, and cRMSE ≈ 0.26. The polymorph-split benchmark yields METRe ≈ 79%, mean RMSE ≈ 0.07, and cRMSE ≈ 0.16 (Martirossyan et al., 15 Sep 2025).

The interpretation given is that models can learn the diversity of perovskite prototypes when polymorphs are not unfairly sequestered in distinct splits. In this framing, the earlier benchmark protocol underestimates model capability because it conflates compositional generalization with exposure to unseen polymorphs of seen compositions.

This has direct implications for benchmark design in crystal structure prediction. The revised protocol assesses out-of-distribution composition generalization while preserving the one-to-many nature of the target distribution. A plausible implication is that perov-5 is most appropriately treated as a benchmark for composition-conditioned polymorph generation, rather than for deterministic single-structure reconstruction.

An associated misconception is that a high composition overlap between train and test necessarily makes a benchmark easier. In the reported perov-5 setting, the opposite can occur: train/test composition overlap under a random split can make evaluation less fair because the held-out item is often a distinct polymorph rather than a near-duplicate.

6. Traffic Perov-5: the autonomous-driving dataset sharing the name

In autonomous-driving research, Perov-5 denotes a processed car-following dataset extracted from the open Lyft Level-5 data. The source dataset consists of 20 self-driving test vehicles equipped with LiDARs and cameras. For each actor and the AV itself, the toolkit provides position $_3$6, yaw angle, bounding-box size, and speed for perceived HVs, while no direct speed for the AV is provided and must be derived from position. Scenes are approximately 25 s long, with a typical inter-frame interval of $_3$7 s, reaching 0.42 s in worst frames (Li et al., 2023).

The extracted car-following data are divided into H-A segments, where a human-driven follower follows the instrumented AV, and H-H segments, where a human-driven follower follows a human-driven leader. Pair selection uses a two-step screening adapted from Wen et al. (2022). Group-1 requires, among other constraints, passenger-car classification probability $_3$8, longitudinal gap $_3$9 m, lateral offset $_3$0 m, no other agents occluding the gap, same side of upcoming traffic lights, straight road segment, and continuous validity for $_3$1 s. Group-2 adds frame-level constraints including yaw deviation $_3$2 rad, max yaw off lane direction $_3$3 rad, $_3$4 s, adjacent-position displacement $_3$5 m, and average speed of both vehicles $_3$6 m/s.

Quality assessment follows Punzo et al. (2011) through kinematic-constraint checks: $_3$7 m/s$_3$8, $_3$9 m/s$C=\{c_1,\ldots,c_n\}$0, and a JSI rule under which jerk’s sign may invert at most once per 1 s. The raw data exhibit substantial anomaly levels, especially in JSI. The paper also identifies a missing-data artifact: 0 m/s at the start/end of each 25 s scene, with values unreliable for the first 0.5–1.5 s after any zero-speed.

Enhancement proceeds in three stages. Missing segments are filled by 7th-degree jerk-minimization, with

$C=\{c_1,\ldots,c_n\}$1

subject to boundary constraints on $C=\{c_1,\ldots,c_n\}$2, $C=\{c_1,\ldots,c_n\}$3, and $C=\{c_1,\ldots,c_n\}$4, while minimizing

$C=\{c_1,\ldots,c_n\}$5

The data are then processed by Kalman filtering using a constant-speed model for AV speed derivation and a constant-acceleration model for over-smoothed acceleration estimation. Finally, wavelet denoising is applied to $C=\{c_1,\ldots,c_n\}$6 using Daubechies-6 (“db6”), 4 levels, and soft thresholding with noise level $C=\{c_1,\ldots,c_n\}$7. Vehicle-length cleaning fixes AV length = 4.87 m and regularizes HV length through variance-based averaging or clamping to [3.5, 6.5] m followed by a 95th percentile estimate.

The published dataset schema includes scene_id, track_id_follower, track_id_leader, t, x_leader, x_follower, v_leader, v_follower, a_leader, a_follower, jerk_follower, spacing, time_headway, length_leader, and length_follower. After enhancement, anomaly rates are reported as H-A: accel. 0.0082%, jerk 0.0039%, JSI 0.455% and H-H: accel. 0.0234%, jerk 0.0186%, JSI 0.454%. The regime analysis uses seven categories: Fa, Fd, C, A, D, F, S. In H-A, A+D+F ≈ 74% of total CF time; in H-H, the corresponding share is ≈70%. Moreover, 64% of H-A and 56% of H-H segments contain A, D, F simultaneously, and are described as ready for IDM calibration.

The final counts are 29,449 H-A segments and 42,892 H-H segments, with total CF duration ≈ 460 h. The detailed summary reports total distance covered ≈ 15,000 km, whereas the abstract reports 150k+ km. The resource is stated to be publicly available at https://github.com/RomainLITUD/Car-Following-Dataset-HV-vs-AV, together with Python scripts for loading, filtering, and visualizing desired subsets. This alternate usage of the name “Perov-5” is therefore a distinct traffic-flow dataset rather than a crystal-structure benchmark.