Phlowers Dataset: Multi-Material Dynamics Benchmark

• Phlowers dataset is a multi-view video database capturing heterogeneous dynamics in flowers, enabling evaluation of material segmentation and physical parameter estimation.
• It employs synchronized RGB videos, ChArUco-based calibration, and SAM2-generated masks to accurately document bending, twisting, and spatially-varying deformations.
• Benchmark results using metrics like PSNR, IoU, and Chamfer Distance demonstrate state-of-the-art performance for multi-material dynamic analysis.

The Phlowers dataset is a multi-view video database designed to serve as the first real-world multi-material dynamics testbed for evaluating algorithms that estimate material composition and continuum-mechanical parameters of physical objects directly from video. Introduced in “M-PhyGs: Multi-Material Object Dynamics from Video” (Wada et al., 18 Dec 2025), Phlowers contains synchronized multi-view RGB videos of human interactions with flowers in controlled settings. The dataset’s primary focus is on capturing the complex, spatially-varying dynamics of natural flowers undergoing bending, poking, and twisting, thereby breaking the assumption of homogeneity prevalent in prior single-material physical parameter estimation benchmarks.

1. Motivation and Scope

Phlowers was motivated by the lack of realistic, multi-material object datasets for visual estimation of mechanical properties. Most existing benchmarks assume objects are composed of a single material or have simplistic topologies, which is not representative of real-world scenarios. Flowers serve as an archetypal case: they routinely comprise at least three distinct material types—e.g., woody stems, fleshy petals, and leathery sepals—integrated into a single, contiguous object with intricate geometry. This heterogeneous composition, combined with their nonrigid, compliant mechanics and familiar form factors, renders flowers particularly suitable for stress-testing multi-material physical inference approaches (Wada et al., 18 Dec 2025).

The stated purpose of Phlowers is to evaluate methods that jointly (i) segment an object into regions with distinct mechanical properties and (ii) estimate parameters such as Young’s modulus ($E$) and density ($\rho$) per region, using only visual input. Unlike prior datasets, Phlowers is designed to capture non-uniform, time-dependent deformations in response to human manipulation.

2. Data Collection Protocol

Each sample in Phlowers documents a controlled physical interaction sequence in which a person arranges or inserts a single cut flower into a flower-frog (pin-holder) fixed on a tabletop. This action imposes complex, multi-axis loads on the flower, naturally eliciting bending, twisting, and spatially-varying deformation in stems and petals. The dataset does not define separate “shake” or “poke” categories; only the insertion/arrangement scenario is considered.

Video acquisition employs five hardware-synchronized cameras. Detailed camera parameters, both intrinsic and extrinsic, are estimated using the COLMAP structure-from-motion framework. Physical alignment of the multi-view setup is achieved via ChArUco boards, enabling a globally consistent coordinate system. Hardware time-coding is used to synchronize frames across views. Environmental conditions are kept consistent: indoor tabletop settings with a static background instrumented by ChArUco, and ambient room lighting. The precise frame rate, resolution, and real-time duration are not specified in the publication.

3. Dataset Composition and Structure

Phlowers consists of 10 distinct flower instances, each captured in a single multi-view sequence, with approximately 100 frames per camera per sequence. Each frame contains the following modalities:

• Multi-view RGB images (5 views per frame)
• Foreground object masks, generated offline with the SAM2 segmenter on a per-view basis, serving as pseudo-ground-truth silhouettes
• 2D manual contact-point annotations for each frame, triangulated to yield 3D contact trajectories reflecting human-induced manipulation
• A dense multi-view 3D Gaussian splatting model of the flower’s undeformed rest geometry

No manual “ground-truth” segmentations for material classes are provided. Instead, material regions are inferred algorithmically (e.g., via M-PhyGs) based on clustered DINO and GARField image features, typically resulting in 3–5 segments per flower (e.g., stem, petal, leaf).

The standard file formats for RGB and mask data are not specified, although masks are distributed as per-view PNG bitmasks. 3D Gaussian splatting representations are furnished either in binary or JSON format, following conventions established in prior 3D Gaussian Splatting work.

4. Ground-Truth, Annotations, and Metadata

Phlowers does not provide measured “ground-truth” values for Young’s modulus ($E$) or density ($\rho$). For each segmented material region, these mechanical parameters are estimated by competing algorithms, with Poisson’s ratio ($\nu$) fixed at a typical value of 0.3. The veracity of estimated parameters is assessed indirectly by comparing predicted vs. observed flower dynamics.

Additional metadata include full camera calibrations (intrinsics and extrinsics), ChArUco-based alignment for scale and pose normalization, rest-shape 3D geometric models, triangulated 3D contact-point trajectories, and estimated 6-DOF camera poses. There are no explicit manual material segmentation masks or direct ground-truth for per-region labels in the public version.

5. Dataset Split, Diversity, and Species Representation

Phlowers does not define an official train/validation/test split at the flower-instance level. Instead, experiments typically employ “in-sequence” evaluation—training parameter estimation on the first 50 frames and testing on subsequent 30 frames (e.g., frames 51–80) of the same sequence. “Cross-sequence” protocols involve training on one time-window and testing on another from the same flower. For ablation studies, four of the ten flower instances were held out as a quasi-validation set to support component-wise evaluation.

The dataset includes at least one carnation, a fig blossom, and eight additional common cut-flower species. The number of material segments per flower ranges from three to five. The mean sequence length is approximately 100 frames, and the average number of segments per flower is approximately four; variances are not reported.

6. Evaluation Framework and Benchmarking

Evaluation on Phlowers centers on three primary tasks: (1) Material parameter estimation (Young’s modulus $E$, density $\rho$ per material segment), (2) Material segmentation quality, and (3) Dynamic video prediction, i.e., forecasting how the flower deforms in response to unseen interactions.

Metrics employed include:

• PSNR (Peak Signal-to-Noise Ratio) between rendered and ground-truth RGB frames (higher is better)
• 2D IoU (Intersection over Union) of rendered mask versus SAM2 foreground mask (higher is better)
• 2D Chamfer Distance (CD) computed between rendered silhouette and mask boundary (lower is better)

Table: Summary of Benchmark Results (In-Sequence Future Prediction) [(Wada et al., 18 Dec 2025), Table 4]

Method	PSNR (dB)	IoU (%)	CD (px)
PhysDreamer	16.00	31.9	52.6
OmniPhysGS	15.31	8.2	163.6
Pixie	15.63	22.5	72.0
Spring-Gaus	15.61	6.6	174.7
gs-dynamics	16.13	34.3	40.6
GIC	15.78	9.5	169.6
M-PhyGs (Ours)	18.49	70.6	3.3

These results establish the state-of-the-art for multi-material parameter estimation and dynamics prediction tasks using Phlowers as a benchmark. Baseline methods include PhysDreamer, OmniPhysGS, Pixie, Spring-Gaus, gs-dynamics, and GIC, as referenced by their respective citation venues (Wada et al., 18 Dec 2025).

7. Distribution, Recommended Preprocessing, and Usage

Phlowers is to be made publicly available at the project website: https://vision.ist.i.kyoto-u.ac.jp/research/m-phygs/. The specific license terms are not stated in the referenced publication.

Recommended preprocessing steps are as follows:

1. Camera calibration via COLMAP; align global scale and orientation using ChArUco board detections.
2. Frame synchronization across views using the provided hardware time-code.
3. Optional generation of per-view object masks with SAM2.
4. Reconstruction of 3D rest-state Gaussians from dense multi-view image capture.
5. Triangulation of manual 2D contact-point annotations to yield 3D manipulation trajectories.

Operational details such as exact frame rate, image resolution, and file encodings are not specified. Users are directed to consult the project website and distributed documentation for precise data specifications.

8. Significance and Future Developments

Phlowers advances the field of visual physical parameter estimation by introducing a natural object category with inherent multi-materiality, challenging geometry, and nontrivial dynamics. Its focus on flowers, with characteristic fine-scale parts and soft-rigid composite behavior, enables rigorous validation of algorithms that move beyond the homogeneous-object assumption. A plausible implication is that future versions of Phlowers or derived datasets may expand the species, material types, or interaction protocols, and introduce direct measurements of mechanical parameters for absolute ground-truth validation. Continued community benchmarking on Phlowers is likely to catalyze algorithmic advances in model-based and data-driven physical inference methodologies (Wada et al., 18 Dec 2025).