---
title: 'RealMat: Diffusion, Benchmarks & Matrix Theory'
url: https://www.emergentmind.com/topics/realmat
type: topic
---

# RealMat: Diffusion, Benchmarks & Matrix Theory

RealMat is a research label used in several technically distinct ways. In current arXiv literature it most explicitly denotes a diffusion-based text-to-material generator for physically based rendering workflows and, in the related name RealMat-BaG, a benchmark for experimentally grounded semiconductor bandgap prediction. In parallel, the same label is used informally for the real \((\beta=1)\) specialization of matrix-variate statistical distributions, for real-matrix problems in linear algebra, for exact real arithmetic and real radical computation, and for Real or real-algebraic structures on matrix-valued categories and rings [2509.01134][2604.25568][1402.5178][1011.5087][1802.06712][1112.3197][1106.5239][2204.13645].

| Usage of RealMat | Central object | Principal source |
|---|---|---|
| Generative graphics | Text-to-material diffusion model for SVBRDF maps | [2509.01134] |
| Materials ML evaluation | Experimental bandgap benchmark with OOD splits | [2604.25568] |
| Matrix-variate statistics | Real \((\beta=1)\) T, T-Riesz, and beta type II laws | [1402.5178], [1011.5087] |
| Linear algebra | Real matrices as linear combinations of orthogonal matrices | [2606.01519] |
| Exact and algebraic computation | Expression-DAG exact real arithmetic; real radicals via moment matrices | [1802.06712], [1112.3197] |
| Real algebra and categories | Orderings on \(M_n(R)\); Real matrix factorizations | [1106.5239], [2204.13645] |
| Data systems analogue | Relational matrix algebra in a column store | [2004.05517] |

## 1. RealMat as a text-to-material generator

In its most explicit contemporary usage, RealMat is a **text-to-material** generator that maps a natural-language prompt to a full set of **PBR/SVBRDF maps**—including albedo or base color, height, roughness, and metallic maps—that are intended to be physically coherent and visually realistic under natural environment lighting [2509.01134]. The system is built on **Stable Diffusion XL (SDXL)** and encodes material outputs as a single **\(2\times 2\)** RGB grid. In the reported layout, the top-left quadrant is albedo, the top-right is height or displacement, the bottom-left is roughness, and the bottom-right is metallic. At \(1024\times 1024\) resolution, each map therefore occupies \(512\times 512\) pixels.

The training pipeline is explicitly two-stage. First, a pretrained SDXL model is finetuned on **synthetic material maps arranged in \(2\times 2\) grids**, so that the model retains SDXL’s text-alignment and realism priors while learning the joint distribution of material maps rather than ordinary photographs. The denoising objective is given as
\[
\mathbb{E}_{(x_0,c),\,x_t,\,t}\left[\|f_\theta(x_t,t,c)-x_0\|^2\right],
\]
and sampling uses a DDPM Markovian step
\[
p_\theta(x_{t-1}\mid x_t,t,c)=\mathcal{N}\big(a_t x_t + b_t f_\theta(x_t,t,c),\,\sigma_t^2 \mathbf{I}\big).
\]
The paper states that this stage used a synthetic SVBRDF corpus derived from approximately **8,615 material graphs**, **~126k variations**, and **~800k render+SVBRDF pairs**, with **AdamW**, learning rate \(2\cdot 10^{-6}\), batch size **120**, and **400k** iterations on **24×80GB A100 GPUs**.

The second stage addresses the remaining realism gap by **reinforcement learning**. RealMat uses **DDPO (Denoising Diffusion Policy Optimization)**, interpreting the denoising trajectory as a policy and maximizing
\[
J(p_\theta)=\mathbb{E}_{\tau\sim p_\theta}[R(\tau)],
\]
with policy gradient
\[
\nabla_\theta J(p_\theta)=\mathbb{E}_{\tau\sim p_\theta}\left[\sum_{t=T}^1 R(\tau)\,\nabla_\theta \log p_\theta(x_{t-1}\mid x_t,t,c)\right].
\]
The reward is computed only on the final sample:
\[
R(\tau)\coloneq r_\phi\big(g(d(x_0),L)\big),\qquad L\sim \mathcal{U}(\mathbf{L}),
\]
where \(d\) is the SDXL VAE decoder, \(g\) is a **Mitsuba 3** rendering pipeline, and \(L\) is drawn from **200** HDR environment maps from **Poly Haven**.

The realism reward model is itself learned. RealMat collects a large-scale dataset of realistic material images under natural lighting, uses **CLIP** image features, and applies a single **linear layer** to produce a scalar realism score. The training loss is
\[
\mathcal{L}_r=\lambda_1\|r_\phi(I)-l_{gt}\|_2+\lambda_2\mathcal{L}_{tv}(\mathrm{CLIP}(I)),
\]
with \(\lambda_1=1\) and \(\lambda_2=100\). The RL finetuning stage uses **LoRA** of rank **4**, **50** denoising steps, learning rate \(3\cdot 10^{-4}\), batch size **128**, and **110** epochs on **32×80GB A100 GPUs**.

Empirically, the paper reports that RL finetuning improves realism. In one user study with **641 responses**, materials **after finetuning** were preferred in **46.8%** of judgments, materials **before finetuning** in **28.9%**, and **24.3%** were marked similar. In a comparative study with **440 responses** against **MatFuse** and **ReflectanceFusion**, **RealMat** was preferred in **53%** of responses, compared with **40%** for ReflectanceFusion, **3%** for MatFuse, and **4%** marked similar. The paper also notes limitations: prompt precision remains challenging, the \(2\times 2\) layout imposes a resolution–channel trade-off, training is computationally expensive, and the realism reward inherits dataset and lighting biases.

## 2. RealMat-BaG and realistic materials evaluation

A second explicit use of the label is **RealMat-BaG**, where “RealMat” denotes a broader benchmark family for **real-world evaluation in materials machine learning**, and “BaG” denotes **bandgap** prediction [2604.25568]. RealMat-BaG is framed as a deployment-centric benchmark for **semiconductors** under **experimental labels**, **cross-fidelity transfer from DFT to experiment**, **domain-aware out-of-distribution splits**, and **interpretability**.

The computational side of the benchmark uses **60,218** filtered **Materials Project** entries with PBE bandgaps and DFT-relaxed structures. The experimental side is an open-access dataset of **1,705** experimental bandgaps aligned to Materials Project structures through MPIDs. These **1,705** samples are reported as **1,183** from BandgapDatabase1, **185** from DS2, and **337** from Matbench-Expt. Bandgap ranges are excluded, multiple measurements are aggregated by the **median**, and when several MP polymorphs share the same reduced formula the entry with the **lowest 0 K formation energy** is selected.

The benchmark defines several evaluation regimes. A random split reserves **10%** of the experimental dataset, or **171 samples**, as a held-out test set, with the remaining **1,534** used for fine-tuning and smaller sub-regimes at **10%**, **25%**, and **50%** of that fine-tuning pool. A feature-based OOD split uses **CGCNN** descriptors and **k-means** with \(k=8\), designating the cluster whose centroid is farthest from the global mean as the OOD test set; this OOD subset contains **189 materials**. A category-based **leave-one-material-out** split filters the data to **1,254 crystals** distributed across categories such as chalcogenides, oxides, halides, antimonides, and silicides. Additional domain splits are defined over **chemical systems**, **periodic groups**, and **crystal systems** using **MatFold**.

The compared models include **Linear Regression**, **SVR**, **Random Forest Regression**, and several GNNs: **CGCNN**, **CartNet**, **ALIGNN**, **CHGNet**, and **LEFTNet**. Metrics are **MAE**, **MRAE**, and \(R^2\). Under the random split, the best classical baseline reported is **SVR** with **MRAE \(=0.261\pm 0.007\)**, **MAE \(=0.510\) eV**, and **\(R^2=0.497\)**. Among pretrained GNNs, **CHGNet** reaches **MRAE \(=0.242\pm 0.004\)**, **MAE \(=0.486\) eV**, **\(R^2=0.487\)**, and **LEFTNet** reaches **MRAE \(=0.242\pm 0.003\)**, **MAE \(=0.485\) eV**, **\(R^2=0.507\)**.

The benchmark’s central result is not simply that some models perform well, but that **generalization degrades sharply under realistic shifts**. Errors increase under feature OOD and domain-based splits, and the paper highlights especially difficult categories such as **silicides** and **antimonides**. It further reports that pretraining on PBE data can improve data efficiency and some OOD settings, yet can also **propagate systematic PBE biases**. This makes RealMat-BaG less a leaderboard for absolute accuracy than a stress test of fidelity mismatch, domain robustness, and the interpretability of materials ML models.

## 3. RealMat as the real case of matrix-variate probability theory

In matrix-variate statistics, “RealMat” is used informally for the **real case** of theories developed uniformly over real normed division algebras, obtained by setting the algebra parameter to \(\beta=1\) [1402.5178][1011.5087]. In this usage, matrices have entries in \(\mathbb{R}\), \(T^\ast T\) becomes \(T^\top T\), and the relevant symmetry groups are the familiar real orthogonal and Stiefel groups.

One strand concerns the **generalised matricvariate \(T\)-distribution**, more specifically the **matricvariate T-Riesz** distribution. In the real case, a central type I law on \(T\in\mathbb{R}^{n\times m}\) has density proportional to
\[
\big|I_m+T^\top T\big|^{-(v+n)/2}\,
q_{-k-t}\big(I_m+T^\top T\big)\,
q_t\big(T^\top T\big),
\]
with parameter conditions
\[
\Re(v/2)>\frac{m-1}{2}-k_m,\qquad
\Re(n/2)>\frac{m-1}{2}-t_m.
\]
The construction replaces the normal/Wishart pair of the classical matrix \(T\) law by **Kotz-Riesz** and **Riesz** distributions, in type I or type II variants. The paper states that the real T-Riesz can be constructed as \(T=X\,U(U)^{-1}\), where \(X\) is real Kotz-Riesz distributed and \(U\) is real Riesz distributed, and that setting \(k=t=0\) recovers the classical real matricvariate \(T\) law. It also reports that \(T^\top T\) has a **matricvariate beta-Riesz type II** distribution, and that the family is orthogonally invariant and admits an affine location–scale form.

A second strand studies the distinction between **matricvariate \(T\)** and **matrix multivariate \(T\)** laws in the real case. The real central matricvariate \(T\) density is given as
\[
f_T(T)=
\frac{\Gamma_m\!\left(\frac{n+v}{2}\right)}
{\pi^{mn/2}\Gamma_m\!\left(\frac{v}{2}\right)|\Sigma|^{v/2}|\Xi|^{m/2}}
\left|I_n+\Xi^{-1}(T-\mu)^\top\Sigma^{-1}(T-\mu)\right|^{-(n+v)/2},
\]
whereas the matrix multivariate \(T\) uses a trace kernel,
\[
f_{T_1}(T_1)=
\frac{\Gamma\!\left(\frac{v+mn}{2}\right)p^{mn/2}}
{\pi^{mn/2}\Gamma\!\left(\frac{v}{2}\right)|\Sigma|^{m/2}}
\left[1+\operatorname{tr}\Big(\Sigma^{-1}(T_1-\mu)^\top(T_1-\mu)\Big)\right]^{-(v+mn)/2}.
\]
The same paper derives the associated beta type II laws obtained from \(TT^\top\) or \(T_1T_1^\top\), and gives joint densities of singular values featuring the standard Vandermonde factor \(\prod_{i<j}(\delta_i^2-\delta_j^2)\). A recurrent point across both papers is that “RealMat” here does **not** denote a separate probabilistic object; it denotes the real specialization of a general algebraic framework.

## 4. RealMat in linear algebra: sums of orthogonal matrices

In real matrix analysis, the label is used for structural questions about arbitrary real square matrices. Pang’s 2026 result resolves the question, recorded by Zhan, of how many real orthogonal matrices are needed in a real linear combination to represent an arbitrary \(A\in M_n(\mathbb{R})\) [2606.01519]. The main theorem states that for every \(n\ge 3\),
\[
M_n(\mathbb{R})=\mathcal{A}_n(3),
\]
where
\[
\mathcal{A}_n(k)=\{\alpha_1Q_1+\cdots+\alpha_kQ_k:\alpha_i\in\mathbb{R},\ Q_i\in O(n)\}.
\]
Equivalently, every real \(n\times n\) matrix with \(n\ge 3\) is a real linear combination of **three** real orthogonal matrices.

The paper sharpens this to an exact minimal classification:
\[
k_{\min}(1)=1,\qquad k_{\min}(2)=2,\qquad k_{\min}(n)=3\quad(n\ge 3).
\]
This improves the earlier **Li–Poon** bound of four orthogonal matrices and matches the lower bound from **Li–Zhang–Li**, which showed that two orthogonal matrices do not suffice for all \(n\ge 3\).

The proof combines **SVD**, **Householder reflections**, a **rank-one update lemma**, and a singular-value characterization of matrices that are combinations of two orthogonal matrices. The strategy is to reduce an arbitrary matrix to a nonnegative diagonal matrix via SVD, show that any such diagonal matrix is a linear combination of three orthogonal matrices, and then conjugate back. For \(n=2\), the decomposition is explicit:
\[
\operatorname{diag}(s_1,s_2)=\frac{s_1+s_2}{2}I_2+\frac{s_1-s_2}{2}\operatorname{diag}(1,-1),
\]
which immediately yields a two-term representation after left and right multiplication by the orthogonal factors of the SVD.

This result is exact rather than asymptotic: three orthogonal matrices are not merely sufficient but **necessary** in dimensions at least three. A plausible implication is that the paper identifies a minimal “orthogonal basis template” for the whole space \(M_n(\mathbb{R})\).

## 5. Exact real arithmetic and real radical computation

A computational use of the label appears in exact arithmetic and polynomial system solving. In the exact-arithmetic setting, RealMat is described as a system or library for **exact real arithmetic or real algebraic computation**, centered on the number type **`Real_algebraic`** and a multithreaded **expression-DAG** evaluation model [1802.06712]. The design follows the **Exact Geometric Computation** paradigm: cheap floating-point interval filters are used first, and if they do not certify a decision, the system refines a shared expression DAG using arbitrary precision arithmetic and algebraic separation bounds.

The paper describes a node type `mtdag_node<Policies>` storing approximations, error bounds, a requested precision, interval filters, separation bounds, parent lists, and an atomic `dependency_count`. Recomputation proceeds in topological order rather than through recursive descent, and parallelism is managed by a singleton `Thread_manager` using `boost::lockfree::queue<std::function<void()>*>`, `std::atomic<unsigned int> thread_count`, and `std::atomic<unsigned int> task_count`. The implementation uses `MAX_THREADS = 4` and `TASK_THRESHOLD = 5`, and schedules node recomputation through atomic `try_recomputation()` calls once child dependencies are resolved. The reported motivation is not symbolic exactness for its own sake, but exact **decisions** such as sign tests and comparisons.

A second algebraic-computational meaning concerns the computation of **real radicals** of polynomial ideals through **moment matrices**, **border bases**, and **semidefinite programming** [1112.3197]. The paper assumes the real variety is finite and combines border basis algorithms with dual methods based on positive semidefinite moment or Hankel matrices. Its central claim is that the resulting algorithm can compute a border basis of the input ideal and can also compute the quotient structure of the **real radical ideal directly**, without prior Gröbner-basis computation. In this framework, a positive finite-rank Hankel operator corresponds to a finite conic combination of point evaluations at real roots, and the kernel of such an operator is a **real radical ideal**. The algorithmic synthesis is therefore: border bases provide numerically stable quotient computations, while moment matrices and SDP restrict the computation to real roots and remove multiplicities.

Taken together, these two papers use “RealMat” to denote real-number exactness in two complementary senses: exact arithmetic over expression DAGs and exact extraction of the real solution set of polynomial systems.

## 6. Real algebra, Real matrix factorizations, and noncommutative reality conditions

In real algebraic geometry for matrix rings, the relevant object is \(M_n(R)\) over a commutative unital ring \(R\), with positivity considered on the symmetric part \(S_n(R)\) [1106.5239]. The paper defines **quadratic modules**, **preorderings**, and **orderings** in \(M_n(R)\), extends the **Artin–Lang theorem** and the **Krivine–Stengle Positivstellensätze** from \(R\) to \(M_n(R)\), and proves that orderings on \(M_n(R)\) are in one-to-one correspondence with orderings on \(R\) via the induction map
\[
\operatorname{Ind}(N)=\{A\in S_n(R): v^\top A v\in N\ \forall v\in R^n\}.
\]
At the same time, the paper emphasizes that this correspondence **fails for preorderings**, so the theory is **not Morita equivalent** to the scalar theory. That non-equivalence is one of the paper’s central structural claims.

A categorically different, but terminologically related, use is the theory of **Real matrix factorizations** [2204.13645]. Starting from a potential \(w\in R\), classical matrix factorizations are \(\mathbb{Z}/2\)-graded free \(R\)-modules with odd differential \(d_M\) satisfying \(d_M^2=w\cdot \mathrm{id}_M\). The paper introduces both **antilinear** and **contravariant** Real versions, organized by **Real 2-representation theory** for a \(C_2\)-graded finite group. In the antilinear case the key category is
\[
\mathrm{MF}^{\mathrm{Real}_G}(R,w)=\mathrm{MF}(R,w)^G,
\]
the homotopy fixed point category of a Real action. In the contravariant case the equivariant matrix factorization category \(\mathrm{MF}_G(R,w)\) acquires a canonical **dg duality structure**, aligning the theory with **Grothendieck–Witt theory** and with the parity symmetries of **Landau–Ginzburg orientifolds**.

The main theorems are Real analogues of **Knörrer periodicity**. In the antilinear setting, the paper proves a Real equivariant quasi-equivalence after adding a quadratic form \(y^2+z^2\), structurally analogous to **\((1,1)\)-periodicity for \(KR\)-theory**. In the contravariant setting, it proves quasi-equivalences of dg categories with duality after adding \(y^2+z^2\) or, more canonically, \(u_1v_1+u_2v_2\), structurally analogous to **\(4\)-periodicity for Grothendieck–Witt theory**. The framework also incorporates finite-group equivariance and **discrete torsion twists**.

A common misconception would be to treat these algebraic usages as mere reformulations of ordinary matrix theory over \(\mathbb{R}\). The literature instead uses “Real” in Atiyah’s and Grothendieck–Witt’s senses: involutive, duality-bearing, and equivariant structures, not just real-number entries.

## 7. Systems-level analogue: relational matrix algebra

At the systems level, work “in the spirit” of a hypothetical RealMat appears in **Relational Matrix Algebra (RMA)**, proposed as a principled logical unification of relations and linear algebra inside a column-store database [2004.05517]. RMA extends relational algebra with matrix operations such as **multiplication**, **inversion**, **QR**, and **SVD**, but remains **closed**: inputs and outputs are always relations, and no separate logical matrix data type is introduced.

The core abstraction splits a relation into an **order schema** and an **application schema**. Matrix constructors \(\mu_{\mathbf{U}}(r)\) and \(\overline{\mu}_{\mathbf{U}}(r)\) extract ordered relational views corresponding to row identities and numerical payload, while a relation constructor \(\gamma(m,\mathcal{R})\) turns matrix rows back into tuples. On that basis, unary operators such as
\[
\mathrm{inv}_{\mathbf{U}}(r)=
\gamma\big(\mu_{\mathbf{U}}(r)\circ INV(\overline{\mu}_{\mathbf{U}}(r)),\ \mathbf{U}\circ\overline{\mathbf{U}}\big)
\]
and binary operators such as matrix multiplication are defined entirely relationally. The paper proves **matrix consistency** for the resulting operators and formalizes **row origins** and **column origins**, which preserve contextual information across operations such as transpose or covariance.

The implementation target is **MonetDB**, where relations are stored as columnar **BATs** and matrix operations can be executed either natively over BATs or via **Intel MKL** after physical conversion to contiguous arrays. The reported experiments show that this design performs well on mixed workloads involving joins, aggregations, regression, covariance computation, and element-wise matrix operations.

Although RMA is not itself called RealMat, it is directly relevant to the broader research pattern captured by that label: it treats matrices over real-valued data as first-class objects without abandoning schema, provenance, or relational optimization. In that sense it is a systems-level counterpart to the mathematical and statistical RealMat usages surveyed above.

Source: https://www.emergentmind.com/topics/realmat