Neural-GASh: Real-Time Shading Pipeline

• The paper demonstrates a real-time shading pipeline that learns per-vertex spherical harmonic coefficients from CGA-encoded geometry, eliminating costly offline precomputation.
• It integrates a 5-layer MLP with Unity’s Barracuda to predict 27 SH coefficients per vertex, enabling dynamic texture swaps and accurate lighting rotation in animated scenes.
• Empirical results show that Neural-GASh achieves sub-33 ms updates for meshes up to ~1,000 vertices, closely matching traditional PRT performance with significantly reduced computational overhead.

Searching arXiv for the specified paper and nearby context. {"query": "\"Neural-GASh\" arXiv", "max_results": 5} Neural-GASh is a real-time shading pipeline for 3D meshes that leverages a neural radiance field architecture to perform image-based rendering using Conformal Geometric Algebra (CGA)-encoded vertex information as input. It replaces the costly offline precomputation of Precomputed Radiance Transfer (PRT) by learning, in a single feed-forward pass, the per-vertex spherical harmonic (SH) radiance coefficients directly from CGA-encoded geometry. Integrated into the Unity engine, it is designed for accurate shading of animated and deformed 3D meshes, with dynamic texture swaps and environment map rotation, and it is evaluated both on conventional meshes and on scenes generated via 3D Gaussian splats (Geronikolakis et al., 18 Jul 2025).

1. Pipeline definition and computational scope

Neural-GASh is organized around the runtime sequence

Input Geometry $\rightarrow$ CGA Encoding $\rightarrow$ Neural Radiance Field (MLP) $\rightarrow$ SH Coefficients $\rightarrow$ Fragment Shader $\rightarrow$ Final Shaded Mesh (Geronikolakis et al., 18 Jul 2025).

A Unity mesh provides per-vertex positions $\mathbf{v}$ and normals $\mathbf{n}$. For each $(\mathbf{v},\mathbf{n})$, the system constructs a 32-dimensional multivector $M$, and a neural field predicts 27 SH coefficients $\{P_{i,0},\dots,P_{i,26}\}$ for vertex $\rightarrow$0. These coefficients are then projected against SH lighting coefficients and consumed by a GPU fragment shader to render the final image in real time.

The central distinction from traditional PRT is explicit: Neural-GASh does not rely on expensive offline precomputations. Instead, the learned model directly consumes CGA-based representations of vertex positions and normals. Because CGA encoding and neural inference are only re-run when the mesh deforms or lighting changes, the per-frame cost remains low. This suggests that the method is structured around event-driven updates rather than unconditional per-frame recomputation.

2. Conformal Geometric Algebra representation

The geometric formulation is stated in 5D CGA. A 3D point $\rightarrow$1 is embedded as a null vector

$\rightarrow$2

Here $\rightarrow$3 and $\rightarrow$4 are the null basis vectors with signature $\rightarrow$5 (Geronikolakis et al., 18 Jul 2025).

For Neural-GASh, the operative input is not merely a point lift or plane lift, but a motor $\rightarrow$6 that encodes both rotation and translation. The construction is given in four steps: compute the unit quaternion $\rightarrow$7 rotating the fixed up-axis $\rightarrow$8 to the normal $\rightarrow$9; build the corresponding CGA rotor $\rightarrow$0; build the CGA translator

$\rightarrow$1

and multiply them to obtain the 32D motor

$\rightarrow$2

This 32-dimensional multivector simultaneously captures local position and orientation, and it is used as the sole input to the neural field. A plausible implication is that the CGA encoding functions as an inductive bias: the network is not asked to reconstruct pose-related structure from raw Euclidean coordinates alone, but receives a representation in which translation and orientation are already algebraically coupled.

3. Neural radiance prediction and optimization

The predictor is a 5-layer MLP $\rightarrow$3. Its input dimension is 32, corresponding to the CGA motor $\rightarrow$4; its hidden layers are fully-connected layers of size 1024, 512, 256, and 128; its activations are SiLU after each hidden layer; and its output is 27 real values, corresponding to SH coefficients $\rightarrow$5, that is, 9 per RGB channel for 3 bands (Geronikolakis et al., 18 Jul 2025).

Regularization consists of batch normalization on inputs and hidden layers, together with dropout with decaying rate. During training, the objective is the mean squared error between network outputs and ground-truth PRT coefficients. At inference time, the predicted coefficients are combined with pre-computed SH lighting coefficients $\rightarrow$6, derived from an HDR skybox, via the inner product

$\rightarrow$7

where $\rightarrow$8 is the number of SH basis functions, with $\rightarrow$9 bands and $\rightarrow$0.

This formulation fixes the output space to low-order SH radiance coefficients rather than directly predicting pixels or fragment-space colors. That choice ties Neural-GASh to a radiance-transfer representation that remains compact and compatible with standard shader pipelines.

4. Spherical harmonics rotation and lighting model

Neural-GASh implements scene shading in Unity, where rotation of scene lights in terms of spherical harmonics is also performed optimally using CGA. To rotate a lighting environment by a CGA rotor $\rightarrow$1, the paper states that one can apply the adjoint action on the embedded directions or directly rotate the SH coefficients via the Wigner-D matrices (Geronikolakis et al., 18 Jul 2025):

$\rightarrow$2

The corresponding geometric action is written as

$\rightarrow$3

An important implementation nuance is that the Unity realization substitutes quaternions for CGA rotors, while the resulting rotation of SH coefficients is described as mathematically equivalent. This addresses a likely misconception: the method is CGA-based in representation and formulation, but not every engine-side operation is executed through an explicit CGA rotor implementation.

The lighting model therefore combines learned per-vertex transfer with SH-domain environment lighting. This suggests that Neural-GASh occupies a middle ground between classical radiance-transfer pipelines and learned radiance fields: it uses a neural predictor, but its runtime lighting composition remains analytically structured.

5. Engine integration and runtime behavior

The trained PyTorch model is exported to ONNX and loaded via Unity’s Barracuda. CGA encoding in C# uses the GA-Unity package to convert $\rightarrow$4 in parallel jobs, writing 32 floats per vertex into a Barracuda tensor (Geronikolakis et al., 18 Jul 2025).

A tensor-layout conversion is required because Barracuda expects $\rightarrow$5 whereas PyTorch uses $\rightarrow$6; the implementation reorders dimensions so that $\rightarrow$7, the vertex count, maps to $\rightarrow$8. For each mesh, an independent Barracuda worker runs the MLP and produces $\rightarrow$9. SH projection is then executed by a compute shader that performs the projection equation in parallel across vertices and assembles color buffers.

The animation and deformation path is explicit. Whenever the mesh deforms or the vertex-normal buffer changes, the CGA encoding $\rightarrow$0 network $\rightarrow$1 shader pipeline re-executes. If motion is rapid, defined as velocity above threshold, updates are skipped to save CPU time, since lighting changes become imperceptible. The fragment shader supports dynamic texture swaps and environment map rotation without restarting the application.

The platform notes delimit the practical operating regime. On mobile/VR hardware, maintaining sub-33 ms updates requires meshes of $\rightarrow$2 vertices; higher counts or multiple meshes will require further optimization or low-rank approximations. Quality-of-service can be traded off by skipping updates on fast motion or by culling distant geometry.

6. Empirical results and comparative capabilities

The evaluation compares Neural-GASh to a Python reference PRT on five meshes of increasing complexity, timing only the precomputation/inference stage. All timings were run on AMD Ryzen 5 2400G + GTX 1060 (6 GB) (Geronikolakis et al., 18 Jul 2025).

Mesh Name	Vertices	PRT Time (s) / NGASh Time (s)
Teapot	672	100.12 / 0.03
Pawn	4 892	868.42 / 0.12
Humanoid	5 340	892.28 / 0.20
Wolf	5 559	962.27 / 0.15
Statue	49 671	10 957.75 / 1.31

The stated real-time threshold is $\rightarrow$3 ms per update, and Neural-GASh achieves this for meshes up to $\rightarrow$4 vertices. Across the five meshes, the MSE, together with standard deviation, between ground-truth PRT coefficients and network predictions ranges from $\rightarrow$5 to $\rightarrow$6. Visually, Neural-GASh matches PRT creases, soft shadows, and low-light regions far more closely than a Blinn-Phong baseline.

The method was also integrated with a 3D Gaussian Splatting scene of approximately 80 k vertices split into 2 submeshes. In that setting, environment map swaps complete in 0.3–0.6 s, versus $\rightarrow$7 s for conventional PRT on the same geometry. This suggests that the pipeline’s flexibility extends beyond conventional mesh benchmarks, although the update latency for that larger scene is not in the sub-33 ms regime.

A compact comparison of stated capabilities is as follows:

Method	Animations/Deformations	Engine Integration
Traditional PRT	No	No
DeepPRT	Yes	No
Neural PRT (image-based)	No	No
3D Gaussian Splatting	No	No
Neural-GASh	Yes	Unity (Barracuda)

The full capability table in the source also lists dynamic textures and lighting rotation. In that comparison, Neural-GASh is the only listed method marked “Yes” for animations/deformations, dynamic textures, and lighting rotation, while also reporting engine integration as “Unity (Barracuda).”

7. Interpretation, limits, and relation to prior practice

Neural-GASh is positioned directly against traditional PRT. The central claim is not merely that inference is fast, but that the pipeline removes offline PRT precomputation while preserving high rendering quality and supporting animated and deformed 3D meshes (Geronikolakis et al., 18 Jul 2025). In that sense, its novelty lies in combining CGA-encoded geometry, SH-domain radiance prediction, and a Unity runtime path.

Several boundaries are explicit. The reported comparison is against a Python reference PRT and measures only the precomputation/inference stage. The real-time claim is qualified by mesh size: it is tied to meshes up to approximately $\rightarrow$8 vertices under the stated threshold. For larger geometries, the paper points to further optimization or low-rank approximations. A plausible implication is that Neural-GASh should be understood as a practical learned shading pipeline with a clearly bounded operating envelope rather than as an unconditional replacement for all PRT workloads.

Another likely misconception concerns precomputation. Neural-GASh eliminates offline PRT precomputation, but its inference still combines predicted transfer coefficients with pre-computed SH lighting coefficients derived from an HDR skybox. This does not contradict the main claim; it clarifies that the removed cost is the geometry-dependent PRT precompute, not every preprocessing step in the lighting pipeline.

In summary, Neural-GASh demonstrates that encoding vertices and normals in CGA and predicting SH coefficients via a lightweight MLP can support deformation-aware global illumination in a production engine without offline PRT precomputation, while retaining compatibility with dynamic textures, lighting rotation, and real-time shader execution.