SPGen: Modular Systems in Vision & Hardware

• SPGen is a modular system that transforms high-level, structured inputs into optimized outputs for computer vision, 3D shape generation, scanpath simulation, and FPGA-based stream processing.
• In vision applications, SPGen leverages multi-scale convolutional aggregation and binary cross-entropy loss to generate high-confidence segmentation prompts, improving metrics like the Dice score.
• For 3D and hardware domains, SPGen employs spherical projection for detailed mesh generation and automates stream processor design on FPGA, reducing resource usage and development time.

SPGen refers to distinct technical systems and algorithmic modules found in computer vision, hardware design automation, and computational neuroscience applications. Each SPGen instance is domain-specific yet united by a core emphasis on structured, modular construction and efficient representation or generation of complex information, such as prompts, 3D surfaces, scanpaths, or arithmetic pipelines.

1. SPGen in Vision: Self-Prompt Generation in UN-SAM

In digital pathology, UN-SAM’s SPGen module facilitates fully automated nuclei segmentation without manual annotation by synthesizing high-confidence segmentation prompts directly from multi-scale visual embeddings. The SPGen module operates after the domain-adaptive ViT encoder, leveraging a hierarchy of spatially aggregated features via convolutional heads at varying resolutions.

Mathematically, input image embeddings $\{h_\ell\}_{\ell=1}^L$ are projected into multi-scale 2D feature maps $F_\ell$ using convolutions with stride $s_\ell$, then fused by feature pyramid network (FPN) top-down aggregation: $$P_L = F_L, \quad P_\ell = F_\ell + \mathrm{Up}(P_{\ell+1}), \quad H_{ms} = P_1$$ A $1\times1$ conv head $C_{\mathrm{prom}}$ produces per-spatial-token logits $G$, and a sigmoid-thresholded binary filter identifies high-confidence mask hints: $$\hat{g}_n = \begin{cases} g_n & \sigma(g_n) \ge \tau \ 0 & \text{otherwise} \end{cases}$$ SPGen is trained via auxiliary binary cross-entropy: $$L_{\mathrm{SPGen}} = -\frac{1}{N} \sum_{n=1}^N \left[y_n \log \sigma(g_n) + (1-y_n)\log(1-\sigma(g_n))\right]$$ These generated prompts are directly incorporated into the query-enhanced decoder for segmentation, eliminating external prompt annotation and empirically improving Dice metrics and domain generalization (Chen et al., 2024).

2. SPGen for 3D Generation: Spherical Projection Generator

SPGen’s role in single-image 3D shape generation is characterized by its use of an injective spherical projection (SP) representation. The approach maps each 3D point $P=(x,y,z)^\top$ onto an equirectangular coordinate $F_\ell$0 plus depth $F_\ell$1, supporting arbitrarily nested or non-watertight topologies via multi-layer stacking: $F_\ell$2 where $F_\ell$3, $F_\ell$4, $F_\ell$5.

Multi-layer SP maps $F_\ell$6 encode the ordered set of surface intersections per ray, accommodating nontrivial interiors. This image-domain parameterization enables leveraging powerful 2D latent diffusion priors; a SPGen system first trains a VAE to encode SP maps, then a conditional U-Net diffusion backbone with explicit layerwise self-attention. This results in enhanced geometric consistency, reduced resource use, and higher fidelity by common metrics (Chamfer, IoU, F-score) over both watertight and open-domain surfaces (Zhang et al., 16 Sep 2025).

3. SPGen for Stochastic Scanpath Generation in Art

SPGen also denotes a deep neural system for simulating human scanpaths—sequences of visual fixations—on paintings. The architecture consists of a MobileNet V2 encoder, learnable Gaussian priors (encoding center/outward bias), a sequence of convolutional merging blocks, and differentiable fixation selection modules employing weighted Soft-ArgMax: $F_\ell$7 Random noise injections model the stochasticity of human gaze ($F_\ell$8, $F_\ell$9 sampled), and domain adaptation is handled by a gradient reversal layer with adversarial loss: $s_\ell$0 where $s_\ell$1 is the binary cross-entropy for domain discrimination and $s_\ell$2 combines BCE for scanpath points with a sequence length loss.

SPGen establishes state-of-the-art performance across benchmarks (MultiMatch, NSS, congruency) and is the first end-to-end system enabling cross-domain prediction for paintings without labeled gaze data in the target domain (Kerkouri et al., 25 Feb 2026).

4. SPGen in Hardware: Stream Processor Generator on FPGA

In the context of hardware design, SPGen is an automated generator for pipelined floating-point stream processors targeting FPGAs. Developers write signal-processing kernels in a high-level, domain-specific language (SPD). The SPGen system constructs a static data-flow graph from SPD, synthesizes pipeline stages, and auto-inserts delay elements to ensure synchronization:

• SPD supports single-assignment IEEE-754 “equ” nodes (generated by FloPoCo) and user-defined HDL modules.
• The back-end explores pipeline depth versus resource trade-offs by varying FloPoCo synthesis frequency.
• Outputs encapsulate Avalon-ST interfaces with synchronized, fully pipelined data paths supporting initiation interval $s_\ell$3.

Benchmarks show SPGen achieves comparable performance (GFLOPS) to hand-designed pipelines but with lower LUT, DSP, and BRAM usage, and dramatically reduced development time for applications such as Lattice Boltzmann fluid dynamics (Sano et al., 2014).

5. Comparative Features of SPGen Implementations

Application Domain	Input/Output Type	Key Architectural Traits
Vision (UN-SAM)	ViT features → mask hints	Multi-scale conv, FPN, BCE loss
3D Generation	Image → SP map → mesh	Spherical 2.5D rep, diffusion priors
Scanpath Generation	Image → scanpath	FCNN, priors, stochastic selector
Hardware (FPGA)	SPD → Verilog core	DFG parsing, pipelining, delays

SPGen instantiations share the design principle of translating high-level, structured or hierarchical input into optimized, domain-specific generative or predictive outputs, often centering structural induction, loss-guided selection/filtering, and integration into broader automated pipelines.

6. Experimental Evidence and Operational Outcomes

SPGen modules yield quantifiable improvements across all reported domains:

• In UN-SAM, SPGen elevates mean Dice scores by approximately 2%, eliminating the need for manual prompting and reducing false positives (Chen et al., 2024).
• For single-view 3D generative modeling, SPGen achieves lower Chamfer distance and higher IoU at reduced GPU-memory and compute budgets relative to multiview diffusion methods (Zhang et al., 16 Sep 2025).
• In stochastic gaze modeling, SPGen outperforms baselines on both MultiMatch and NSS, and adapts to artistic domains without requiring new gaze labels, supporting flexible, artifact-cognizant museum applications (Kerkouri et al., 25 Feb 2026).
• For FPGA-based HPC applications, SPGen’s SPD-based automation delivers comparable or better resource efficiency while drastically improving developer productivity (Sano et al., 2014).

7. Limitations and Future Directions

Documented constraints include the lack of feedback-loop (cycle) support in hardware DFGs, fixed initiation intervals in stream processing, potential limits of SPGen’s layer count for highly complex geometric interiors, and the need for further work on higher-level SPD compilers or multi-modal prompt conditioning. Planned extensions in several domains involve more sophisticated data-flow patterns (hardware), robust cross-modality adaptation (art/vision), and scaling structured generative models (geometry).