H-Reducer: Efficient Model & Feature Aggregation

• H-Reducer is a suite of methodologies that aggregate features and project model states to achieve computational efficiency across disciplines.
• Hyper-reduced autoencoders employ stencil subsampling, collocation, and Gappy-POD to drastically lower online costs while preserving accuracy.
• Petrov–Galerkin and interpolation-based H∞/H2 methods enable scalable reduced-order models with significant speedups and robust error minimization.

The term "H-Reducer" encompasses several distinct model reduction and feature aggregation methodologies developed in diverse domains, including nonlinear dynamical systems, control theory, computer vision, and network systems. This article presents a comprehensive synthesis of the concept by drawing on established constructions in projection-based reduced order modeling, $H^\infty$-optimal and $H^2$-optimal interpolation, vision-to-text aggregation, and Petrov–Galerkin hyper-reduction.

1. Overview and Definitions

"H-Reducer" denotes methods designed to achieve significant computational and representational efficiency by selective aggregation, collocation, interpolation, or parameterization, typically with the goal of preserving essential system structure and accuracy in surrogate or reduced models. The term appears both in scientific machine learning—where it refers to hyper-reduced autoencoders or Petrov–Galerkin surrogates for nonlinear PDEs—and in visual document understanding, where it designates a feature map aggregation module for efficient multimodal processing. In classical model reduction, H-Reducer is also used for interpolation-based $H^\infty$ and $H^2$-optimal state-space reductions.

2. Hyper-Reduced Autoencoders for Nonlinear Model Reduction

In the context of nonlinear PDE-constrained optimization and scientific computing, the H-Reducer is a "hyper-reduced autoencoder" methodology for projection-based model reduction on nonlinear manifolds (Cocola et al., 2023). The high-level procedure comprises the following steps:

1. Latent Manifold Representation: Given a time-discrete full-order model (FOM) with state $\mathbf{u}^n \in \mathbb{R}^N$, the classical Manifold-LSPG ROM projects trajectories onto a low-dimensional latent space $\mathbf{y}^n \in \mathbb{R}^k$ via a decoder map $\mathbf{g}: \mathbb{R}^k \rightarrow \mathbb{R}^N$.
2. Stencil Subsampling: H-Reducer departs from global approximation by focusing only on a stencil mesh (subset of DOFs), with a reduced decoder $$\hat{\mathbf{g}}: \mathbb{R}^k \rightarrow \mathbb{R}^{n_s}$$ such that $$\hat{\mathbf{g}}(\mathbf{y}^n) \approx \mathbf{S}_s \mathbf{u}^n$$ for selection matrix $\mathbf{S}_s$.
3. Noisy Autoencoder Training: The encoder $\hat{\mathbf{f}}$ introduces Gaussian noise to promote robustness, with parameters learned over subsampled snapshots restricted to the stencil mesh.
4. Collocation-Based Hyper-Reduction: The reduced-order solution at time step $n$ is computed by minimizing the collocated residual

$$\mathbf{y}^n = \arg\min_{\mathbf{y} \in \mathbb{R}^k} \left\| \mathbf{S}_c \mathbf{R}^n\big(\hat{\mathbf{g}}(\mathbf{y}), \dots\big) \right\|_2^2,$$

where $\mathbf{S}_c$ is the sample mesh selection matrix for residual entries.

1. Gappy-POD Reconstruction: The full state $\tilde{\mathbf{u}}^n$ is recovered via Gappy Proper Orthogonal Decomposition using

$$\tilde{\mathbf{u}}^n = \bm{\Psi} \hat{\mathbf{u}}_s^n, \qquad \bm{\Psi} = \bm{\Phi}_r (\mathbf{S}_s \bm{\Phi}_r)^\dagger,$$

with $\bm{\Phi}_r$ the POD modes of the FOM snapshots.

This procedure achieves both drastic offline and online cost reductions; only stencil mesh DOFs are visited, and end-to-end full-state surrogates are obtained without ever assembling global residuals (Cocola et al., 2023).

3. Interpolatory H-infinity Model Reduction

The H-Reducer also refers to an interpolatory $H^\infty$-optimal reduction algorithm for large-scale state-space systems (Flagg et al., 2011). The goal is to construct a strictly proper, real rational ROM $H_r(s) = c_r^\top (sI - A_r)^{-1} b_r + d_r$ minimizing the $H^\infty$ norm error between $H$ and $H_r$. Key steps are:

1. H\textsubscript{2}-Optimal Point Selection: Use IRKA to identify $r$ interpolation points (mirror images of ROM poles) and compute Hermite conditions $$H(-\hat{\lambda}_i) = H_r^0(-\hat{\lambda}_i), \; H'(-\hat{\lambda}_i) = H_r^0{}'(-\hat{\lambda}_i)$$.
2. d-Term Parameterization: Augment the ROM with a one-parameter family

$$H_r(s, d_r) = H_r^0(s) + d_r \frac{(G_1(s) - 1)(G_2(s) - 1)}{1 - d_r G_3(s)},$$

enabling the enforcement of $2r+1$ interpolation (nearly equioscillating $|H-H_r|$ on the imaginary axis).

1. Loewner Matrix Surrogates: Avoid full-order $H^\infty$-norm optimization via data-driven Loewner pencils, constructing surrogate error systems for efficient scalar parameter optimization.

These techniques yield scalable, near-optimal $H^\infty$ model reduction for systems of dimension $n \sim 10^5$, with accuracy typically exceeding that of balanced truncation and cost dominated by sparse solves as in IRKA (Flagg et al., 2011).

4. Vision-to-Text Aggregation in Document Understanding

In vision-LLMs, the H-Reducer is a feature reduction module for efficient layout-preserving aggregation of transformer vision model outputs (Hu et al., 19 Mar 2024):

• Module Design: Given a ViT grid $X \in \mathbb{R}^{H \times W \times C}$, the H-Reducer applies a $1\times r$ convolution with stride $1\times r$ along the horizontal axis, merging $r$ adjacent columns into a single feature, producing $X' \in \mathbb{R}^{H \times W' \times C}$, $W' = W/r$.
• Layout Preservation: Row ordering and block-wise horizontal positions remain intact; this is critical for downstream structure-aware LLMs, as no vertical information is lost.
• Sequence Efficiency: By aggregating along width, the visual token length is reduced from $L=H \times W$ to $L' = H \times W/r$ without compromising document or table layout.
• Implementation: A convolutional kernel followed by a linear projection aligns the aggregated visual features to the LLM's embedding space. Variant kernel sizes were evaluated; $(1\times 4)$ is optimal for most document/text tasks.

This approach yields a substantial reduction in processing costs and memory for high-resolution document images with negligible loss in structural fidelity (Hu et al., 19 Mar 2024).

5. Petrov–Galerkin and Hyper-Reduction for Reduced Models

In nonlinear finite-element model reduction, the H-Reducer denotes an alternative to both Galerkin and LSPG projection for Petrov–Galerkin ROMs (Parga et al., 2023):

• Fixed Left Basis Construction: The method builds an iteration-invariant left basis $\Psi$ via SVD of projected residuals or Jacobians, avoiding the parameter- and time-dependent construction of LSPG.
• Projected Assembly: Residual and Jacobian computations are performed element-by-element without complementary mesh patches, as $\Psi$ enables direct local assembly:

$\Psi^\top R(u) = \sum_{e=1}^L \Psi_e^\top R^e(u),$

thus substantially reducing online cost and not requiring mesh patching.

• Empirical Cubature Hyper-Reduction: ECM selects sample elements and weights to assemble projected quantities at cost $\mathcal{O}(\ell)$, $\ell \ll L$.
• Numerical Effectiveness: For both SPD and non-SPD Jacobians, the Petrov–Galerkin H-Reducer achieves accuracy on par with LSPG and Galerkin, with reduced measurement set sizes and significant speedups, e.g., $>100\times$ for large structural mechanics benchmarks.

This strategy ensures residual minimization while keeping the reduced system assembly strictly local, with scalability independent of the full-model size (Parga et al., 2023).

6. $H^2$-Optimal Model Reduction for Positive Networks

For positive network systems, H-Reducer refers to an $H^2$-optimal reduction strategy using Riemannian augmented Lagrangian optimization under positivity and structural constraints (Misawa et al., 2021):

• Parametrization: The reduced dynamics matrix is written as $A_r = (J_r - R_r) Q_r$, with $J_r$ skew-symmetric, $R_r$ and $Q_r$ symmetric positive-definite, ensuring $A_r$ is Hurwitz.
• Objective: The $H^2$-error $\|G - \widehat{G}\|_{H^2}$ is minimized:

$J = 2 F(A_r, B_r, C_r) + \|G\|_{H^2}^2,$

where $F$ is computable via Lyapunov/Sylvester equations for various state covariance matrices.

• Constraints: Nonnegativity and pattern preservation are imposed on $(A_r, B_r, C_r)$ per initial clustering estimates, with equality/inequality constraints handled in the Lagrangian.
• Riemannian Gradient: Gradients are computed respecting the product manifold geometry of the parameters; updates are performed via Riemannian CG/LS.
• Performance: On synthetic benchmarks, the H-Reducer achieves errors $\mathcal{E}_2 = 3.14\%$ and $\mathcal{E}_\infty = 4.67\%$ compared to clustering baselines at $\sim 70\%$, fully preserving positivity and reducing dynamic order (Misawa et al., 2021).

7. Summary Table: H-Reducer Methodologies Across Domains

Context	Core Principle	Notable Feature / Result
Nonlinear PDE Surrogates (Cocola et al., 2023)	Hyper-reduced autoencoder, collocation, stencil mesh, Gappy-POD	Offline/online costs $\mathcal{O}(n_s)$, accuracy beyond linear POD
State-Space Systems (Flagg et al., 2011)	Interpolatory $H^\infty$ reduction (IRKA + d-term + Loewner)	Near-optimal $H^\infty$ error, only sparse solves, $n \to 10^5$ feasible
Vision-LLMs (Hu et al., 19 Mar 2024)	Horizontal aggregation via convolution	$4\times$ token reduction with layout preserved
Petrov–Galerkin ROMs (Parga et al., 2023)	Fixed left basis Petrov–Galerkin + ECM	No patch mesh, local assembly, $100\times$ speedup
Positive Network Models (Misawa et al., 2021)	$H^2$-error minimization under Riemannian constraints	Guarantees positivity/structure, $>20\times$ error reduction

Each method labeled H-Reducer addresses the dual challenge of drastic computational savings and accuracy preservation by leveraging specialized aggregation, projection, or interpolation strategies structurally adapted to the domain and target constraints.