---
title: 'RieszNet: Dual Neural Architectures'
url: https://www.emergentmind.com/topics/riesznet
type: topic
---

# RieszNet: Dual Neural Architectures

RieszNet denotes two distinct but influential architectures in neural network research: (1) a multitask neural network used for automatic debiasing and Riesz representer estimation in statistical machine learning and causal inference [2110.03031, 2409.19777, 2508.17576], and (2) a scale-invariant convolutional neural network designed for image analysis, leveraging the Riesz transform’s mathematical properties to achieve scale equivariance in a single network pass [2305.04665, 2501.18376]. Both approaches exploit the analytic structure of the Riesz transform or representation to encode theoretically desirable properties—orthogonality and statistical efficiency in the first line, provable scale equivariance in the second.

## 1. Mathematical Foundations

### 1.1 Riesz Representer in Statistical Learning

The Riesz representer arises from the Riesz representation theorem, central to estimation of linear functionals on Hilbert spaces. For data \(W=(Y,Z)\), given an outcome regression function \(g_0(z) = \mathbb{E}[Y|Z=z]\) and a linear target functional
\[
\psi_0 = \mathbb{E}[m(W;g_0)],
\]
there exists a unique function \(\alpha_0(Z)\) such that
\[
\psi_0 = \mathbb{E}[\alpha_0(Z)g_0(Z)].
\]
Learning \(\alpha_0\) transforms complex functionals into tractable inner products, crucial for constructing unbiased (doubly robust) estimators [2110.03031, 2409.19777, 2508.17576].

### 1.2 Riesz Transform in Signal Processing

For a signal \(f\) in \(L_2(\mathbb{R}^d)\), the first-order Riesz transform \(\mathcal{R}_j\) (spatial domain operator) is defined by
\[
\widehat{\mathcal{R}_j[f]}(\omega) = -i \frac{\omega_j}{\lVert\omega\rVert} \hat f(\omega),
\]
where \(\hat f\) is the Fourier transform. Key properties critical to RieszNet for images include:

- Exact scale equivariance: for the dilation operator \(L_a[f](x) = f(x/a)\),
  \[
  \mathcal{R}_j[L_a[f]](x) = L_a[\mathcal{R}_j[f]](x).
  \]
- Translation equivariance and steerability.
- All-pass filtering across spatial frequencies [2305.04665, 2501.18376].

## 2. RieszNet for Automatic Debiasing and Causal Effect Inference

RieszNet’s multitask neural network implementation addresses bias in plug-in estimation of linear functionals in high-dimensional settings. Specifically, it learns a joint representation for both the outcome regression \(g(Z)\) and the Riesz representer \(\alpha(Z)\).

### 2.1 Architecture

- **Shared Backbone:** Typically multilayer perceptrons or pre-trained LLM encoders (e.g., DistilBERT for text).
- **Two Heads:** One head outputs the regression \(g(Z)\approx\mathbb{E}[Y|Z]\), the second outputs \(\alpha(Z)\approx\alpha_0(Z)\).
- **Optional:** Additional heads or backbone variants (frozen/unfrozen for stability) [2508.17576].

### 2.2 Losses and Estimation

The core loss jointly penalizes:

- Prediction error of \(g(Z)\) (cross-entropy or squared loss).
- Riesz MSE loss for \(\alpha(Z)\):
  \[
  L_\text{Riesz} = \mathbb{E}[(\alpha(Z)-\alpha_0(Z))^2] \propto \mathbb{E}\big[-2m(W;\alpha) + \alpha(Z)^2\big]
  \]
- Regularization and (optionally) causal-effect matching penalties.

The final estimator for the linear functional (e.g., ATE) is the doubly robust (DR) form:
\[
\widehat\psi_\mathrm{DR} = \frac{1}{n} \sum_{i=1}^n \left[m(W_i;g) + \alpha(Z_i)(Y_i-g(Z_i))\right]
\]
which is Neyman orthogonal and possesses double robustness [2110.03031, 2409.19777, 2508.17576].

#### Empirical Performance

CausalSent [2508.17576], a RieszNet instantiation, achieved 2–3× reduction in mean absolute error of effect estimates relative to propensity-based methods on semi-synthetic IMDB sentiment data. Ensembling over seeds/hyperparameters further stabilizes and improves estimation.

## 3. RieszNet for Scale-Invariant Image Segmentation

A separate line of work introduces RieszNet as a convolutional neural network with inherent scale equivariance for detection and segmentation in images and volumes, notably for cracks in concrete [2305.04665, 2501.18376].

### 3.1 Architecture

- **Riesz Layer:** Replaces spatial convolutions with linear combinations of first- and second-order Riesz transforms of each feature map.
- **No Pooling:** Receptive field expands via repeated Riesz transforms, obviating spatial downsampling.
- **BatchNorm + ReLU:** Used after each Riesz layer; both operations preserve scale equivariance.
- **Minimal Parameter Count:** Example: 3D RieszNet with ~7,000 parameters vs. 2M for 3D U-Net [2501.18376].

### 3.2 Theoretical Guarantee

If the input is scaled by any factor \(a>0\), the output is simply the scaled response of the original network:
\[
\text{RieszNet}\{f(\cdot/a)\} = \big(\text{RieszNet}\{f\}\big)(\cdot/a).
\]
This property holds for arbitrary compositions of RieszNet layers with scale-commuting nonlinearities and normalization [2305.04665, 2501.18376].

### 3.3 Applications and Results

- **Crack Segmentation:** Achieves Dice coefficients 0.90–0.96 across a wide range of crack widths, greatly surpassing standard U-Net generalization outside the training scale.
- **MNIST Large Scale:** RieszNet trained at a single scale generalizes well to unseen scales (accuracy ≈98.5% for scales [0.5,2.0], >80% even at scale 8 with simple padding) [2305.04665].

## 4. Training Protocols and Implementation

### 4.1 Causal Inference (Automatic Debiasing)

- Optimization with Adam gradient methods.
- Losses jointly minimized over regression and Riesz representer branches, with empirical risk over minibatch distributions.
- Early stopping and regularization crucial to prevent Riesz-head instability, especially due to the unboundedness of the Riesz MSE loss [2409.19777, 2508.17576].
- Ensemble multiple models for MAE and bias reduction [2508.17576].

### 4.2 Scale-Invariant Perception

- Generated semi-synthetic training data using fractional Brownian surfaces or Voronoi minimum-weight surfaces, with precise crack masks for supervised training [2501.18376].
- Augmentation with random rotations, zooms, blurring, and gray-value perturbations.
- Weighted binary cross-entropy to address class imbalance.
- Training feasible on limited annotated data due to low parameter count and inbuilt scale generalization [2305.04665, 2501.18376].

## 5. Extensions and Variants

### 5.1 Moment-Constrained RieszNet

Moment-constrained learning further stabilizes the Riesz representer, introducing explicit constraints to the empirical moment of the learned Riesz functional, greatly improving robustness to hyperparameter settings [2409.19777].

### 5.2 Domain-Specific Adaptations

- **Causal NLP:** RieszNet-style architectures (CausalSent) enable interpretable, doubly robust treatment effect estimation at scale with text models, isolating the causal impact of tokens through the learned \(\alpha(Z)\) [2508.17576].
- **Fiber-Reinforced Concrete:** RieszNet’s architecture is theoretically advantageous for structures with complex backgrounds; results suggest adaption to fiber types requires minimal calibration [2501.18376].

## 6. Comparative Assessment and Outlook

RieszNet-type architectures unify practical multitask neural design with rigorous mathematical properties—Neyman orthogonality, double robustness, and scale equivariance. In causal effect estimation, they outperform propensity-based and functional regression competitors in finite-sample and high-dimensional settings, both in simulation and case studies (e.g., impact of "love" on movie review sentiment, establishing a +2.9% causal effect substantially below naive association) [2508.17576].

In perception tasks, RieszNet achieves parameter efficiency and provable generalization beyond training distribution scales—segmenting real and synthetic cracks, or classifying digits at unknown magnifications—without multi-scale pyramids or scale augmentation [2305.04665, 2501.18376].

Embeddings of RieszNet into group equivariant or moment-constrained frameworks offer promising future improvements. Potential limitations include lack of rotational equivariance (in image RieszNets) and, for Riesz-heads, the risk of instability if unconstrained or improperly regularized. Generalization to entirely novel domains (backgrounds, aggregate mixes, fiber types) and adaptation to 3D/4D imaging remain active directions [2305.04665, 2501.18376, 2409.19777].

Source: https://www.emergentmind.com/topics/riesznet