FruitSplat: 3D Bruise Inspection in DexFruit

• FruitSplat is a 3D bruise inspection method that reconstructs fruit pre- and post-manipulation to quantitatively measure visible surface damage using Gaussian Splatting.
• It integrates ordinary RGB video, COLMAP pose estimation, and 2D segmentation cues (Grounded SAM 2 and YOLOv12) to embed bruise likelihood directly into a unified 3D representation.
• The approach computes a bruise score as the percentage of bruised surface points, offering a quantitative evaluation of manipulation outcomes, particularly for strawberries.

Searching arXiv for the cited FruitSplat-related papers to ground the article in current literature. I’m retrieving the relevant arXiv records for FruitSplat and closely related Gaussian-splatting fruit papers. FruitSplat is a post-hoc 3D inspection method introduced within the DexFruit framework for measuring visible bruising on fragile fruit, specifically by reconstructing a fruit before and after manipulation and embedding bruise likelihood directly into a 3D Gaussian Splatting representation. It is presented as a response to a practical evaluation gap: bruising is often subtle, spatially localized, curved over a nonplanar surface, and sometimes only partially visible from any one camera view, while existing approaches are either qualitative and subjective or require specialized sensing such as OCT, MRI, spectroscopy, or X-ray. FruitSplat combines ordinary RGB video, COLMAP pose estimation, 3D Gaussian Splatting, and 2D segmentation cues to produce a high-resolution 3D fruit representation in which bruise semantics are stored per Gaussian and can be compared before and after manipulation (Swann et al., 9 Aug 2025).

1. Concept and role within DexFruit

FruitSplat is one of the two components of DexFruit. The first component is a tactile-informed manipulation policy for gently grasping fruit; the second is FruitSplat, which measures visible bruising after manipulation. In that system-level decomposition, FruitSplat is not part of the robot’s control loop. Its function is evaluative: it reconstructs a fruit before and after manipulation and computes a visible-bruising metric in 3D, thereby making manipulation outcomes quantitatively measurable rather than anecdotal (Swann et al., 9 Aug 2025).

The method is explicitly designed around surface-visible damage. The retained points in the final representation correspond to the fruit surface, and the reported metric is the percentage of surface points classified as bruised. The paper does not define an area integral, volumetric bruise estimate, or internal-damage metric for FruitSplat. Internal damage is handled separately in DexFruit through a stiffness test, not through the Gaussian-splatting representation.

The authors position FruitSplat against two common alternatives. Purely image-based bruise detectors can identify damage in individual images, but they do not directly quantify how much of the fruit surface is damaged in a geometrically meaningful way. Specialized sensing methods can probe damage more directly, but they require more expensive hardware. FruitSplat instead aims to provide an accessible, camera-only, spatially grounded measurement of visual surface bruising.

2. Data acquisition and reconstruction pipeline

The FruitSplat pipeline begins with two scans of the same fruit instance: one before manipulation and one after manipulation. For each state, the fruit is imaged by taking a 360-degree video using a simple webcam. The paper states that videos of the entire target strawberry are collected and fed into COLMAP to estimate camera poses and sparse points for the frames (Swann et al., 9 Aug 2025).

In parallel, every frame is processed by two 2D models. Grounded SAM 2 is used to obtain strawberry object masks, and a custom fine-tuned YOLOv12 bruising segmentation model is used to obtain bruise masks. The YOLOv12 model is trained on 2,000 hand-labeled images collected separately from the robot experiments. These 2D masks function as pseudo-ground-truth supervision signals.

FruitSplat is implemented by extending Nerfstudio’s Splatfacto. A 3D Gaussian Splatting model is then trained from the captured multiview RGB frames and the pseudo-label masks. After optimization, the learned Gaussian set is exported. The representation is filtered by the learned strawberry score so that the fruit surface is isolated from background and support structure. Each retained point also carries an encoded bruise score interpreted as a bruise probability.

This 2D-to-3D transfer is central to the method. The strawberry masks supervise which Gaussians correspond to the fruit, while the bruise masks supervise which fruit-surface regions are bruised. Through multiview consistency during optimization, these signals are distilled into a unified 3D representation. The result is not merely a collection of per-view bruise predictions; it is a 3D surface model whose Gaussians explicitly store fruit-membership and bruise-likelihood information.

3. Gaussian representation and optimization

FruitSplat follows the paper’s 3DGS preliminaries. A camera ray is defined as

$$\mathbf{r}(t) = \mathbf{o} + t\mathbf{d}, \quad t \geq 0,$$

where $\mathbf{o}$ is camera origin and $\mathbf{d}$ is ray direction. The rendered color along a ray is computed by alpha compositing ordered Gaussian contributions:

$$\hat{C} = \sum_{i \in N} c_i \alpha_i \prod_{j=1}^{i-1} (1 - \alpha_j),$$

where $c_i$ is the color of Gaussian $i$, and $\alpha_i$ is obtained by evaluating the projected 2D Gaussian footprint with covariance $\Sigma$. The RGB reconstruction loss is

$$\mathcal{L} = (1 - \lambda)\,\mathcal{L}_1 \;+\; \lambda\,\mathcal{L}_{D\text{-}SSIM}.$$

The paper does not provide a more detailed closed-form equation for 3D Gaussian projection to the image plane, nor an explicit rasterization formula for $\alpha_i$ beyond stating that it comes from evaluating a 2D Gaussian with covariance $\mathbf{o}$0 (Swann et al., 9 Aug 2025).

FruitSplat augments each Gaussian primitive with two additional learned semantic attributes, $\mathbf{o}$1 and $\mathbf{o}$2, indicating whether the Gaussian belongs to the strawberry and whether it lies in a bruise region. These quantities are rendered into 2D mask images analogously to color rendering: a strawberry mask from the $\mathbf{o}$3 values and a bruise mask from the $\mathbf{o}$4 values. The semantic outputs are supervised by the pseudo-ground-truth 2D masks produced during preprocessing.

The paper states that bruise and strawberry loss terms are computed using cross-entropy and added directly to the main 3DGS loss; in the preliminaries this is described as supervision done “similar to color ... instead using a binary cross entropy loss.” No explicit symbolic total-loss equation is printed for these semantic terms. An implementation detail emphasized by the authors is the use of stop-gradient during rasterization of bruise and strawberry masks, so that only the semantic parameters $\mathbf{o}$5 and $\mathbf{o}$6 receive gradient from the semantic losses. This is intended to improve training stability and to avoid having semantic supervision distort the geometric or radiance optimization.

A notable architectural property is modularity. The implementation uses Grounded SAM 2 for strawberry masks and the custom YOLOv12 model for bruise segmentation, but the paper states that the 3DGS representation can distill outputs from “any relevant 2D model.” This suggests a model-agnostic semantic interface rather than a design tied specifically to those two front-end networks.

4. Surface bruising metric and output representation

FruitSplat produces both qualitative and quantitative outputs. Qualitatively, it yields high-quality 3D renderings of the strawberry with bruise localization on the surface. Quantitatively, it yields a per-point bruise probability on the strawberry surface and a final bruising score computed as the percentage of strawberry surface points whose bruise score exceeds a preset confidence threshold (Swann et al., 9 Aug 2025).

The metric is operationally defined rather than written as a formal equation. After training, the Gaussian means and semantic values are exported, the representation is filtered by the strawberry score, bruise confidence is thresholded, and the percentage of bruised points is computed. FruitSplat then compares the pre-manipulation and post-manipulation percentages, and their difference is used as the visible bruising metric. The paper repeatedly refers to “percentage of bruised points” and to a “post-manipulation bruise difference.”

This point-based percentage is important for interpreting what FruitSplat does and does not measure. It measures visible surface bruising on the reconstructed fruit surface. It does not claim to estimate internal tissue damage, volumetric bruise extent, or a continuous surface-area integral. The metric is therefore best understood as a surface-semantic measurement derived from a semantically labeled Gaussian surface reconstruction.

Because the method operates on before-and-after scans of the same fruit instance, it supports quantitative comparison of manipulation outcomes for a single specimen. This per-instance differential structure is what allows FruitSplat to function as an inspection instrument within DexFruit, rather than merely as a static 3D bruise visualizer.

5. Empirical use in DexFruit evaluation

FruitSplat is used on strawberries only. The broader DexFruit experiments cover strawberries, cherry tomatoes, and blackberries, but the FruitSplat damage-inspection experiments focus on strawberries because strawberries exhibit clearly visible external bruising; tomatoes showed nearly no visible bruising, and blackberries were not evaluated with this visual metric (Swann et al., 9 Aug 2025).

Across the overall DexFruit study, the authors report over 630 trials total, with 25 trials per strawberry, 50 per tomato, and 20 per blackberry in the manipulation setup. The full system achieves an average 92% grasping success rate, up to a 20% reduction in visual bruising, and up to a 31% improvement in grasp success rate on challenging fruit compared to baselines. For strawberries specifically, the table reports 95% success and 80% retained stiffness for the full method, versus 91% success and 29% stiffness for the vision-only baseline.

The strawberry bruise metric reported in Table I is as follows:

Method	Strawberry bruise metric
Ours	$\mathbf{o}$7
Touch	$\mathbf{o}$8
Vision	$\mathbf{o}$9

The paper summarizes these results as showing that the post-manipulation bruise difference was almost zero for the full method, compared to 20% bruising for the worst baseline. The negative touch-only value is explicitly attributed by the authors to inconsistencies in pseudo-ground-truth bruise mask detection by the YOLO model.

Within the logic of DexFruit, this comparison is one of FruitSplat’s main demonstrations. It exposes a large visible-bruising gap between policies that would be difficult to quantify from isolated RGB frames or subjective inspection alone. The manipulation side of DexFruit reduces damage during grasping, and FruitSplat supplies the camera-only, spatially grounded measurement that verifies the effect.

6. Scope, limitations, and relation to adjacent fruit-centric Gaussian-splatting work

The paper states several limitations for FruitSplat. First, it is currently demonstrated only on strawberries, and extension to arbitrary fruit types is left as future work. Second, the method depends on the quality of pseudo-ground-truth 2D segmentation; the negative bruise percentage in one baseline illustrates sensitivity to segmentation error. Third, the method requires two complete scans—before and after manipulation—so reconstruction and semantic training must be run twice per fruit instance. Fourth, while the approach emphasizes low hardware cost because only a camera is required, the paper does not provide detailed runtime or training-cost numbers. It also does not deeply analyze sensitivity to lighting, view coverage, or pose-estimation quality, although the reliance on RGB video, COLMAP, and multiview 3DGS makes these natural factors (Swann et al., 9 Aug 2025).

Within the broader literature, FruitSplat occupies a specific niche. It is a fruit-surface semantic inspection method for visible bruising, not an active mapping system, an open-vocabulary counting system, or a whole-plant phenotyping pipeline. By contrast, “Active Semantic Mapping of Horticultural Environments Using Gaussian Splatting” emphasizes active viewpoint selection, hybrid semantic Octomap plus 3DGS reconstruction, and downstream fruit counting and volume estimation in horticultural rows (Cuaran et al., 17 Jan 2026). “CountingFruit: Real-Time 3D Fruit Counting with Language-Guided Semantic Gaussian Splatting” stores compressed CLIP-aligned language features per Gaussian and performs prompt-based semantic filtering for orchard-scale fruit counting (Li et al., 1 Jun 2025). “Object-Centric 3D Gaussian Splatting for Strawberry Plant Reconstruction and Phenotyping” focuses on background-free whole-plant reconstruction and plant-trait extraction rather than bruise quantification on individual fruit surfaces (Li et al., 4 Nov 2025).

This comparison clarifies FruitSplat’s specificity. Its novelty lies less in a new low-level Gaussian renderer than in a fruit-damage measurement formulation: ordinary RGB video is converted into a high-resolution 3D surface representation whose Gaussians store strawberry-membership and bruise-likelihood semantics, and visible bruising is quantified as the change in the percentage of bruised surface points before and after manipulation. A plausible implication is that FruitSplat is best understood as an inspection-oriented semantic 3DGS method for fragile-fruit handling studies, rather than as a general-purpose agricultural reconstruction framework.