Stitching and dimensionality effects on large artificially generated volume datasets

Abstract: Generating large images via deep learning requires patching input data to accommodate hardware memory limitations, then assembling output patches, a process that can introduce stitching artifacts when neighboring patches do not align at borders. While these artifacts are known to affect segmentation tasks, their impact on generative models for style-transfer remains poorly understood. We investigated three stitching approaches and two patch dimensionalities (2D vs 3D) using cycleGAN models trained on cryo-electron microscopy datasets. We evaluated both perceptual quality and performance on downstream mitochondria segmentation. Our key findings reveal that: (1) FID scores fail to detect subtle stitching artifacts that significantly impact downstream segmentation performance, (2) 3D models with artifact-free stitching marginally outperform 2D models on downstream tasks, though the improvement barely justifies the computational cost, and (3) 2D models train more stably due to larger batch sizes. Additionally, we demonstrate that ensembling predictions from three orthogonal directions can improve low-quality volumes but provides no benefit for high-quality outputs. These results demonstrate that maximizing generative model performance on large scientific datasets requires careful consideration and mitigation of stitching artifacts, and that perceptual metrics alone are insufficient for evaluating domain adaptation quality in biomedical imaging.

• The paper shows that stitching artifacts propagate to downstream segmentation tasks, even when perceptual FID scores appear optimal.
• It compares tile-and-stitch, padded overlapping, and non-overlap methods, emphasizing the importance of artifact-free assembly in enabling accurate segmentation.
• The study finds marginal segmentation gains for 3D over 2D models, highlighting a critical trade-off between performance improvements and increased computational costs.

Detailed Authoritative Summary of "Stitching and dimensionality effects on large artificially generated volume datasets"

Introduction

The paper "Stitching and dimensionality effects on large artificially generated volume datasets" (2606.20095) comprehensively investigates the impact of patch-based assembly strategies and patch dimensionality in deep generative models for constructing large volume datasets, particularly within the context of cryo-electron microscopy. The authors systematically evaluate the effects of three stitching strategies and two patch dimensionalities (2D and 3D) using cycleGANs trained on EM data, benchmark both perceptual performance (FID) and downstream segmentation accuracy, and analyze the interplay between model architecture, data assembly, and artifact propagation.

Experimental Design

The primary experimental framework involves training cycleGANs to perform rat-to-human style transfer on large EM stacks, followed by generating volumes using various stitching and assembly strategies. For each combination, the authors evaluate volumes at several GAN checkpoints based on FID scores and subsequently assess their suitability for downstream semantic segmentation using Unet architectures.

Figure 1: Overview of experimental setup comparing stitching methods, dimensionality, checkpoint selection by FID, and downstream segmentation evaluation.

The three principal stitching approaches are:

1. Tile-and-Stitch: Valid convolutions with cropping according to downsampling step, designed for artifact-free assembly.
2. Padded Convolutions with Averaged Overlap: Convolutions with padding and overlapping patches assembled via averaging.
3. Valid Convolutions, No Overlap: Non-overlapping patches with no padding, assembled directly.

These methods were applied with both 2D and 3D patch layouts, and an additional "orthoslice" ensemble utilized three orthogonal dimensional assembles. All experiments were evaluated both perceptually (via FID) and for downstream segmentation (via IoU).

Stitching Artifacts and Perceptual Assessment

The study demonstrates that subtle stitching artifacts are propagated into downstream segmentation tasks, even when perceptual metrics (FID) fail to detect them. The presence of border misalignment, residual padding artifacts, and local intensity anomalies is evident through residual difference maps between approaches.

Figure 2: Visualization of stitching approaches and corresponding residual difference maps revealing borderline artifacts undetectable by FID.

While visual inspection and FID fail to flag some artifacts, downstream segmentation masks and probability maps show subtle but significant degradation, especially when patch boundaries intersect regions of interest. The tile-and-stitch method exhibits minimal artifacts; overlapping and padded approaches yield detectable residual lines at patch boundaries.

Downstream Segmentation Impact

Quantitative evaluation reveals that 3D models with artifact-free stitching (tile-and-stitch) marginally outperform 2D models in downstream IoU, yet the improvement is small (typically within the third decimal place). The advantages are most pronounced when binary cross-entropy loss is used to train the downstream Unet, but are diminished with alternate loss formulations.

Notably, 2D models train more stably due to larger feasible batch sizes, and segmentation quality does not directly correlate with perceptual FID—models with optimal FID do not necessarily yield optimal segmentation performance. The propagation of stitching artifacts, especially in non-artifact-free approaches, is visible in segmentation residuals.

Figure 3: Downstream segmentation masks and probability maps, highlighting boundary artifact propagation from input assembly strategies.

Dimensionality and Ensemble Assembly

The dimensionality comparison demonstrates that 3D models offer only marginal improvements over 2D models for downstream tasks. Nonetheless, the increase in computational demand for training and inference with 3D models is significant and may not be warranted in many practical scenarios.

The orthoslice ensemble method, which aggregates predictions from three orthogonal directions, can ameliorate low-quality volume artifacts but provides no benefit over baseline approaches for high-quality outputs. This ensemble method is only feasible for isotropic datasets and is experimentally validated for the first time in the context of GAN-based style transfer.

Figure 4: Segmentation comparison between 2D, 3D, and orthoslice ensemble assembly in tile-and-stitch, with corresponding FID and IoU values.

Effects of Hyperparameter Choices

Analysis on the openorganelle dataset reveals that training stability is improved with larger 2D batch sizes and identity loss inclusion, while patch size variation mainly reduces run-to-run variance. FID scores are inherently noisy in 3D models; the baseline FID between domains sets an upper bound on achievable perceptual improvement.

Figure 5: Evolution of FID across GAN training checkpoints, depicting parameter-dependent convergence and high variance in 3D settings.

Practical and Theoretical Implications

The study contradicts the assumption that perceptual metrics like FID reliably reflect downstream utility in scientific image assembly and emphasizes the necessity of explicitly evaluating generative outputs in domain-relevant tasks (e.g., segmentation). For bioimaging, artifact-free stitching and dimensionality-aware assembly are essential to maximizing generative model value. The findings suggest that practitioners should:

• Use tile-and-stitch artifact-free assembly approaches for best downstream performance
• Prefer 2D models unless computational resources justify marginal 3D gains
• Evaluate generative models by actual downstream task metrics rather than FID alone
• Consider ensemble assembly only for low-quality or artifact-prone models

The authors also provide an open codebase implementing efficient assembly, stitching, and mixed-precision inference for arbitrarily large datasets, enhancing reproducibility and facilitating broader adoption.

Limitations and Future Directions

The segmentation task is restricted to binary foreground-background masks in isotropic EM data. Extension to more complex downstream objectives (e.g., instance segmentation, attention-based models) and evaluation metrics tailored to biomedical domains, rather than natural image statistics (like FID), is critical. Ideally, perceptual metrics derived from relevant biological image embeddings would be developed. As GPU resources evolve, robust 3D training with larger batch sizes may become practical, improving model stability and potential gains.

Conclusion

This analysis of large-volume generative assembly reveals that stitching artifacts, even when visually subtle, propagate into downstream tasks and evade established perceptual metrics. Artifact-free assembly, dimensionality-aware model selection, and explicit downstream task evaluation are mandatory for scientific imaging applications. The practical benefits from 3D models are marginal relative to computational cost, but artifact mitigation—especially via tile-and-stitch strategies—is critical for robust segmentation performance. The research underscores the need for downstream-centric metrics and domain-specific generative benchmarks in bioimage analysis.