---
title: 'Perov-5 Dataset: Materials & Traffic Benchmarks'
url: https://www.emergentmind.com/topics/perov-5-dataset
type: topic
---

# Perov-5 Dataset: Materials & Traffic Benchmarks

Searching arXiv for papers related to "Perov-5 Dataset" to ground the article in current literature.
Perov-5 is a context-dependent dataset designation used in two distinct research areas. In materials informatics, perov-5 denotes a benchmark of perovskite crystal structures used for crystal structure prediction and benchmark design; in autonomous-driving research, Perov-5 also denotes a processed car-following dataset extracted from the Lyft Level-5 Open Dataset. For clarity, the Editor’s terms **“materials Perov-5”** and **“traffic Perov-5”** are used below. The materials benchmark is centered on polymorph-aware evaluation of generative models, whereas the traffic dataset is centered on extracting and denoising human-following trajectories for mixed-traffic analysis [2509.12178], [2305.18921].

## 1. Terminology and domain scope

The same name has been applied to two unrelated datasets. This suggests a naming ambiguity in recent literature and makes domain context essential when interpreting references to “Perov-5.”

| Usage | Domain | Core object |
|---|---|---|
| materials Perov-5 | crystal structure prediction | perovskite unit cells |
| traffic Perov-5 | car-following analysis | CF trajectory segments |

In the materials literature, perov-5 is described as a set of **18,928 perovskite unit cells**, each containing **exactly five atoms**, and originally curated by Castelli et al. from the Materials Project [2509.12178]. In the traffic literature, Perov-5 is a processed car-following dataset derived from the open Lyft Level-5 data and organized into **H-A** and **H-H** trajectory segments [2305.18921].

A common misconception is to treat “Perov-5” as an unambiguous benchmark name. The literature summarized here does not support that assumption. Instead, the same label refers to two technically unrelated resources with different data models, evaluation protocols, and scientific purposes.

## 2. Materials Perov-5: composition, structure count, and polymorphism

The materials Perov-5 benchmark contains **18,928 perovskite unit cells** and **9,646 distinct reduced compositions**. Of these compositions, **9,282 appear exactly twice** as polymorph pairs, while **364 compositions appear only once**. The reported **polymorphism load** is **roughly 98% of compositions are polymorphic**, making perov-5 an **extremely polymorph-rich test set** [2509.12178].

These counts are central to how the benchmark behaves. Because most compositions have more than one structurally distinct entry, the benchmark is not merely a composition-to-structure lookup problem. It is, instead, a one-to-many prediction setting in which multiple legitimate structural realizations may correspond to the same stoichiometry.

The paper illustrates this with explicit polymorph clusters. **CaCdSO\(_2\)** is reported to have **two distorted perovskite prototypes**, with **S and O swapped/enriched**. **HfNbN\(_3\)** is reported to have **two ABX\(_3\) variants** with **Hf and Nb exchanged on the A/B crystallographic sites**. These examples clarify why a benchmark based only on composition identity can obscure structurally meaningful diversity.

A plausible implication is that perov-5 is most informative when used to evaluate structural coverage rather than only single-reference recovery. That interpretation is consistent with the later introduction of polymorph-aware splitting and metrics.

## 3. Random splitting failure and polymorph-aware resplitting

The benchmark critique focuses on the original split attributed to **Xie et al.’s random 60–20–20% division** of the **18,928 structures** into train, validation, and test subsets. In that split, polymorph pairs are frequently separated across subsets. The reported composition overlap is substantial: **2,265 validation structures share compositions with the training set**, and **2,214 test structures share compositions with the training set**. Yet among those overlapped compositions, only **94** validation structures and **107** test structures **“match” within the standard StructureMatcher tolerances**, despite sharing the same stoichiometry [2509.12178].

The stated consequence is that a generative model is **penalized for failing to reproduce a specific polymorph that it was never trained on**, which **artificially depresses the reported match rate**. This is the core methodological objection to random splitting in a highly polymorphic dataset.

To address this, the paper proposes **“perov-5-polymorph-split”**, which treats each reduced composition and all of its polymorphs as an indivisible group. Formally, with \(C=\{c_1,\ldots,c_n\}\) the set of **9,646 unique compositions**, and \(P(c)=\{s\in\text{perov-5}:\mathrm{composition}(s)=c\}\) the polymorph set of a composition, the split partitions \(C=C_{\text{train}}\cup C_{\text{val}}\cup C_{\text{test}}\) so that
\[
|C_{\text{train}}| \approx 0.60|C|,\quad
|C_{\text{val}}| \approx 0.20|C|,\quad
|C_{\text{test}}| \approx 0.20|C|.
\]
The corresponding structure subsets are then defined as
\[
S_{\text{train}}=\bigcup_{c\in C_{\text{train}}} P(c),\quad
S_{\text{val}}=\bigcup_{c\in C_{\text{val}}} P(c),\quad
S_{\text{test}}=\bigcup_{c\in C_{\text{test}}} P(c).
\]

The approximate numerical outcome is **5,788 train compositions, 1,929 validation compositions, and 1,929 test compositions**, corresponding to about **11,576 train structures, 3,857 validation structures, and 3,857 test structures**. By construction, **no polymorph pair is ever split across subsets**. Models trained on \(S_{\text{train}}\) are therefore evaluated on **entirely new compositions**, which may themselves have multiple polymorphs.

## 4. Polymorphism-aware evaluation metrics

The benchmark critique also targets the standard **one-to-one “match rate”**, defined as the fraction of generated structures matching a single reference structure under **StructureMatcher** tolerances. The paper argues that this metric is **blind to polymorphic multiplicity** and therefore mis-specified for one-to-many structure generation [2509.12178].

Two corrected metrics are introduced. The first is **METRe** (**Match-Everyone-To-Reference**) rate. If \(N_{\text{test}}=|S_{\text{test}}|\), and for each reference structure \(i\in S_{\text{test}}\), \(\mathrm{RMSE}_i\) is the minimum StructureMatcher RMSE among all generated structures of the same composition, then a reference is counted as matched if \(\mathrm{RMSE}_i \le \mathrm{stol}\), and
\[
\mathrm{METRe}=\frac{N_{\text{ref.match}}}{N_{\text{test}}}.
\]

The second is **cRMSE** (**corrected RMSE**), defined as
\[
\mathrm{cRMSE}(\mathrm{stol})=
\frac{\sum_{i=1}^{N_{\rm ref.match}} \mathrm{RMSE}_i + \mathrm{stol}\cdot\bigl(N_{\rm test}-N_{\rm ref.match}\bigr)}{N_{\rm test}}
=
\mathrm{METRe}\cdot(\overline{\mathrm{RMSE}}_{\rm matched}-\mathrm{stol})+\mathrm{stol}.
\]
Here, \(\overline{\mathrm{RMSE}}_{\rm matched}\) is the mean RMSE over the matched subset, and **non-matched references are penalized by assigning them an RMSE = stol**.

The stated rationale is twofold. **METRe** allows **each test reference, including each polymorph, to find some generated structure that matches it**, thereby rewarding coverage of polymorphic diversity. **cRMSE** restores a continuous error penalty, combining coverage and positional error so that models cannot improve the metric by matching only a small set of easy references.

## 5. Benchmark behavior and scientific implications

Under the paper’s reported evaluation, the benchmark behavior changes materially when both the split and the metric are made polymorph-aware. The **random-split baseline** on perov-5 yields **METRe ≈ 58%**, **mean RMSE ≈ 0.09**, and **cRMSE ≈ 0.26**. The **polymorph-split** benchmark yields **METRe ≈ 79%**, **mean RMSE ≈ 0.07**, and **cRMSE ≈ 0.16** [2509.12178].

The interpretation given is that models can learn the diversity of perovskite prototypes **when polymorphs are not unfairly sequestered in distinct splits**. In this framing, the earlier benchmark protocol underestimates model capability because it conflates compositional generalization with exposure to unseen polymorphs of seen compositions.

This has direct implications for benchmark design in crystal structure prediction. The revised protocol assesses **out-of-distribution composition generalization** while preserving the one-to-many nature of the target distribution. A plausible implication is that perov-5 is most appropriately treated as a benchmark for **composition-conditioned polymorph generation**, rather than for deterministic single-structure reconstruction.

An associated misconception is that a high composition overlap between train and test necessarily makes a benchmark easier. In the reported perov-5 setting, the opposite can occur: train/test composition overlap under a random split can make evaluation less fair because the held-out item is often a distinct polymorph rather than a near-duplicate.

## 6. Traffic Perov-5: the autonomous-driving dataset sharing the name

In autonomous-driving research, Perov-5 denotes a processed car-following dataset extracted from the open **Lyft Level-5** data. The source dataset consists of **20 self-driving test vehicles equipped with LiDARs and cameras**. For each actor and the AV itself, the toolkit provides **position \((x,y)\)**, **yaw angle**, **bounding-box size**, and **speed for perceived HVs**, while **no direct speed for the AV** is provided and must be derived from position. Scenes are **approximately 25 s long**, with a typical inter-frame interval of **\(\Delta t \approx 0.1\) s**, reaching **0.42 s in worst frames** [2305.18921].

The extracted car-following data are divided into **H-A** segments, where a **human-driven follower follows the instrumented AV**, and **H-H** segments, where a **human-driven follower follows a human-driven leader**. Pair selection uses a two-step screening adapted from **Wen et al. (2022)**. Group-1 requires, among other constraints, **passenger-car classification probability \(>0.95\)**, **longitudinal gap \(<85\) m**, **lateral offset \(<1.75\) m**, **no other agents occluding the gap**, **same side of upcoming traffic lights**, **straight road segment**, and continuous validity for **\(>16\) s**. Group-2 adds frame-level constraints including **yaw deviation \(<0.035\) rad**, **max yaw off lane direction \(<0.087\) rad**, **\(\Delta t<0.42\) s**, **adjacent-position displacement \(<5\) m**, and **average speed of both vehicles \(>1\) m/s**.

Quality assessment follows **Punzo et al. (2011)** through kinematic-constraint checks: \(a\in[-8,5]\) m/s\(^2\), \(j\in[-15,15]\) m/s\(^3\), and a **JSI** rule under which **jerk’s sign may invert at most once per 1 s**. The raw data exhibit substantial anomaly levels, especially in **JSI**. The paper also identifies a missing-data artifact: **0 m/s at the start/end of each 25 s scene**, with values **unreliable for the first 0.5–1.5 s after any zero-speed**.

Enhancement proceeds in three stages. Missing segments are filled by **7th-degree jerk-minimization**, with
\[
x(t)=\sum_{i=0}^{7} p_i t^i,
\]
subject to boundary constraints on \(x\), \(v=\dot{x}\), and \(a=\ddot{x}\), while minimizing
\[
J=\int_0^T [x'''(t)]^2\,dt.
\]
The data are then processed by **Kalman filtering** using a constant-speed model for AV speed derivation and a constant-acceleration model for over-smoothed acceleration estimation. Finally, **wavelet denoising** is applied to \(a_v\) using **Daubechies-6 (“db6”)**, **4 levels**, and **soft thresholding** with noise level \(\sigma=\sigma_a\). Vehicle-length cleaning fixes **AV length = 4.87 m** and regularizes HV length through variance-based averaging or clamping to **[3.5, 6.5] m** followed by a **95th percentile** estimate.

The published dataset schema includes **scene_id**, **track_id_follower**, **track_id_leader**, **t**, **x_leader**, **x_follower**, **v_leader**, **v_follower**, **a_leader**, **a_follower**, **jerk_follower**, **spacing**, **time_headway**, **length_leader**, and **length_follower**. After enhancement, anomaly rates are reported as **H-A: accel. 0.0082%, jerk 0.0039%, JSI 0.455%** and **H-H: accel. 0.0234%, jerk 0.0186%, JSI 0.454%**. The regime analysis uses seven categories: **Fa, Fd, C, A, D, F, S**. In **H-A**, **A+D+F ≈ 74% of total CF time**; in **H-H**, the corresponding share is **≈70%**. Moreover, **64% of H-A** and **56% of H-H** segments contain **A, D, F simultaneously**, and are described as **ready for IDM calibration**.

The final counts are **29,449 H-A segments** and **42,892 H-H segments**, with **total CF duration ≈ 460 h**. The detailed summary reports **total distance covered ≈ 15,000 km**, whereas the abstract reports **150k+ km**. The resource is stated to be **publicly available** at `https://github.com/RomainLITUD/Car-Following-Dataset-HV-vs-AV`, together with **Python scripts for loading, filtering, and visualizing** desired subsets. This alternate usage of the name “Perov-5” is therefore a distinct traffic-flow dataset rather than a crystal-structure benchmark.

Source: https://www.emergentmind.com/topics/perov-5-dataset