Aggregate Value Regression (AVR)

• AVR is a regression framework that relies on aggregate data (e.g., bag means, study summaries, or combined forecasts) to estimate individual or overall responses.
• It employs various likelihood methods—Gaussian for means, logistic for rankings, and plug-in approximations in meta-regression—to model aggregate observations.
• AVR addresses challenges like identifiability and aggregation bias, balancing bias-variance trade-offs in diverse applications from survival analysis to electricity-demand forecasting.

Searching arXiv for the cited papers and related AVR work to ground the article. Aggregate Value Regression (AVR) is a term used in the cited literature for regression procedures in which aggregate information, rather than fully observed individual-level targets, is central to estimation. In "Learning from Aggregate Observations" (Zhang et al., 2020), AVR refers to learning instance-level regression models from bag-level aggregate observations such as means, differences, and ranks. In "Multilevel network meta-regression for general likelihoods: synthesis of individual and aggregate data with applications to survival analysis" (Phillippo et al., 2024), AVR denotes study-level meta-regression that plugs aggregate covariate summaries into a regression model. In "Clustering-based aggregate value regression" (Hirose et al., 21 Aug 2025), AVR denotes a linear forecasting method for aggregate quantities formed by combining many unit-level regression models into a single model. Across these usages, the common theme is regression under aggregation, but the statistical role of the aggregate object differs substantially: it may be the supervision signal, the covariate summary, or the prediction target.

1. Terminological scope and recurring problem structure

The cited papers use the acronym for three distinct settings that share a reliance on aggregate information.

Usage of AVR	Aggregate object	Primary goal
Bag-level regression	Mean, sum, difference, rank, min/max over bags	Predict labels of unseen individuals
Meta-regression with aggregate data	Study-level means, proportions, or other summaries	Adjust treatment effects at the study level
Aggregate forecasting	Sum of unit responses	Forecast aggregate quantities

In the bag-level setting, supervision is given to sets of instances instead of individual instances, while the goal is still to predict labels of unseen individuals. This extends multiple instance learning beyond binary classification to multiclass classification and regression, and it also extends learning from label proportions to real-valued labels and broader aggregate operators. In the meta-regression setting, AVR uses study-level aggregate covariate values—typically means, proportions, or other summaries—to adjust treatment effects at the study level when individual patient data are unavailable. In the forecasting setting, AVR is constructed by combining all regression models into a single model so that estimation is performed directly against an aggregate response rather than against separate unit responses (Zhang et al., 2020).

These formulations are related by a common statistical tension. Aggregation can simplify data collection, preserve privacy, or match how outcomes are naturally observed, but it also changes identifiability, likelihood construction, and the bias-variance profile of the estimator. The main methodological divide is between approaches that derive an aggregate-level likelihood from an individual-level model and approaches that use a plug-in or direct aggregate-loss construction.

2. Bag-level AVR as learning from aggregate observations

In the regression formulation of "Learning from Aggregate Observations," instances $i=1,\ldots,n$ have features $x_i \in X$ and latent real-valued labels $y_i \in \mathbb{R}$. A parametric predictive model $f_\theta: X \to \mathbb{R}$ is posited so that $f_\theta(x_i) \approx y_i$. In the probabilistic formulation, the latent target is modeled conditionally as

$Z \mid X \sim p(Z \mid \theta = f(X;W)),$

where $W$ are the parameters of a deterministic function $f(\cdot;W)$ that outputs $\theta$. For regression with homoscedastic additive Gaussian noise, one convenient choice is

$$Z = f(X;W) + \epsilon,\qquad \epsilon \sim \mathcal{N}(0,\sigma^2).$$

Training data are organized into bags $x_i \in X$0. Instead of observing $x_i \in X$1, one observes an aggregate $x_i \in X$2 generated by an aggregate function $x_i \in X$3: $x_i \in X$4 The cited examples include mean, sum, differences of bag means, and rank comparisons between bags.

Under the aggregate observation assumption

$x_i \in X$5

and the independence assumption

$x_i \in X$6

the bag-level likelihood is

$x_i \in X$7

Given a dataset $x_i \in X$8, the maximum-likelihood objective is

$x_i \in X$9

with estimator

$y_i \in \mathbb{R}$0

This framework accommodates a variety of aggregate observations. For regression, the paper explicitly treats mean observation, difference observation, and rank observation. For classification, the same framework also supports pairwise similarity and triplet comparison. The formulation is deliberately model-agnostic at the instance level: simple maximum likelihood solutions can be applied to differentiable models such as deep neural networks and gradient boosting machines (Zhang et al., 2020).

3. Aggregate likelihoods, gradients, and optimization

For mean observation, the arithmetic mean in a bag $y_i \in \mathbb{R}$1 is modeled as

$y_i \in \mathbb{R}$2

Under $y_i \in \mathbb{R}$3, the predicted bag mean is

$y_i \in \mathbb{R}$4

and the bag-level likelihood is Gaussian: $y_i \in \mathbb{R}$5 Up to a constant, the log-likelihood is

$y_i \in \mathbb{R}$6

Let $y_i \in \mathbb{R}$7. Then

$y_i \in \mathbb{R}$8

For difference observation, two bags $y_i \in \mathbb{R}$9 generate

$f_\theta: X \to \mathbb{R}$0

with

$f_\theta: X \to \mathbb{R}$1

If $f_\theta: X \to \mathbb{R}$2, then

$f_\theta: X \to \mathbb{R}$3

For rank observation, pairwise bag-order constraints are modeled either by a Gaussian difference model or by a logistic likelihood via Gumbel latent scores. In the logistic form,

$f_\theta: X \to \mathbb{R}$4

with log-likelihood

$f_\theta: X \to \mathbb{R}$5

where $f_\theta: X \to \mathbb{R}$6, and gradient

$f_\theta: X \to \mathbb{R}$7

All of these are instances of the unified objective

$f_\theta: X \to \mathbb{R}$8

In practice, mini-batches are formed over bags, bag means are computed by averaging per-instance predictions, bag indices are precomputed, and vectorized gather/scatter operations are used to handle varying bag sizes. Regularization may use standard weight decay; for rank- or difference-only supervision, the cited implementation guidance recommends anchoring with a few direct labels or a mean constraint to fix additive constants, and initialization near zero or a prior mean to stabilize early training.

The reported empirical results use UCI benchmarks with bags formed by grouping 4 instances for regression via mean observation, and pairwise comparisons generated according to the Gaussian rank model for regression via rank observation. Mean-observation AVR achieves mean squared error comparable to fully supervised training and consistently outperforms baselines that naïvely treat bag means as instance labels. Rank-observation AVR, under Gaussian or logistic ranking models, achieves reasonable error variance, typically higher than mean observation but competitive with RankNet-style baselines. Across models, gradient boosting machines generally outperform linear models, and the relative performance ordering is direct $f_\theta: X \to \mathbb{R}$9 mean $f_\theta(x_i) \approx y_i$0 rank (Zhang et al., 2020).

4. Identifiability, consistency, and limitations in bag-level AVR

The theoretical analysis in "Learning from Aggregate Observations" is organized around consistency up to an equivalence relation. Let

$f_\theta(x_i) \approx y_i$1

and define $f_\theta(x_i) \approx y_i$2 iff $f_\theta(x_i) \approx y_i$3. Under mild conditions—compact parameter space, semicontinuity and measurability of the log-likelihood, integrability of local suprema, and asymptotic optimality of the empirical maximizer—the maximum-likelihood estimator converges in probability to the equivalence class $f_\theta(x_i) \approx y_i$4. The stated theorem is a Wald-type consistency result with weakened identifiability.

For mean observation, the paper states that the only transformations preserving the likelihood are exact equality $f_\theta(x_i) \approx y_i$5 almost everywhere. Under Gaussian noise and bag means,

$f_\theta(x_i) \approx y_i$6

so the estimator is consistent in the usual sense.

For difference and rank observations, the likelihood depends only on differences of predictions. Adding a constant $f_\theta(x_i) \approx y_i$7 to all predictions leaves differences and ranks invariant, and the equivalence class becomes

$f_\theta(x_i) \approx y_i$8

The estimator is therefore consistent only up to a global additive constant. If occasional direct labels or a known global mean are available, that constant is fixed.

The framework rests on conditional independence and on a parametric conditional target model $f_\theta(x_i) \approx y_i$9, typically Gaussian in the regression setting. The cited discussion also notes several extensions and caveats. Gaussian noise yields closed-form likelihoods for means and rank via differences. Heavy-tailed noise can be handled by Cauchy-based models, for which mean and difference aggregates remain analytically tractable within the stable family. For count data, Poisson sums give closed-form likelihoods for sum- or mean-type aggregates. Suggested remedies for offset indeterminacy or instability include a zero-mean constraint over the training distribution, a small proportion of direct labels, penalization of prediction variance within bags, or heteroscedastic modeling through prediction of $Z \mid X \sim p(Z \mid \theta = f(X;W)),$0. The main pitfalls identified are offset ambiguity under rank- or difference-only training, loss of resolution under very large bags, and bias when strong within-bag dependence violates the independence assumption (Zhang et al., 2020).

5. AVR in meta-regression: plug-in aggregate covariates and aggregation bias

In network meta-analysis and meta-regression, AVR denotes analyses that use study-level aggregate covariate values—typically means, proportions, or other summaries—to adjust treatment effects at the study level when individual patient data are unavailable. In an individual-level network meta-regression model,

$Z \mid X \sim p(Z \mid \theta = f(X;W)),$1

The AVR plug-in approximation replaces the integral over the covariate distribution $Z \mid X \sim p(Z \mid \theta = f(X;W)),$2 by evaluation at the study mean covariate vector $Z \mid X \sim p(Z \mid \theta = f(X;W)),$3: $Z \mid X \sim p(Z \mid \theta = f(X;W)),$4 and then uses

$Z \mid X \sim p(Z \mid \theta = f(X;W)),$5

The cited paper identifies several sources of aggregation or ecological bias. First, nonlinearity of link functions implies, in general,

$Z \mid X \sim p(Z \mid \theta = f(X;W)),$6

Second, odds ratios, hazard ratios, and survival time ratios are non-collapsible, so marginal and conditional effects differ. Third, treatment-covariate interactions $Z \mid X \sim p(Z \mid \theta = f(X;W)),$7 make the covariate distribution itself relevant to expected outcomes and relative effects. Fourth, different trials may have different covariate distributions $Z \mid X \sim p(Z \mid \theta = f(X;W)),$8, and AVR equates those differences to differences in means, which is insufficient under nonlinear links and interactions.

The proposed remedy is Multilevel Network Meta-Regression (ML-NMR), which integrates the individual likelihood over the aggregate-data covariate distribution rather than plugging in mean covariates. If

$Z \mid X \sim p(Z \mid \theta = f(X;W)),$9

then the aggregate-data marginal likelihood contribution is

$W$0

For binary outcomes this yields

$W$1

and the arm-level marginal likelihood is Binomial. For survival outcomes with time-to-event observation $W$2,

$W$3

and

$W$4

This is evaluated numerically by quasi-Monte Carlo using Sobol’ sequences, with an integration-error diagnostic based on the $W$5 statistic across chains with different numbers of points.

The same framework supports a broad class of survival models, including Exponential, Weibull, and Gompertz proportional hazards, several accelerated failure time forms, and a flexible proportional-hazards baseline hazard based on cubic M-splines: $W$6 The spline coefficients lie on the unit simplex and are regularized through a weighted random walk prior on the inverse-softmax scale, with $W$7.

The reported simulation study compares ML-NMR with full individual-patient-data analysis in a two-trial Weibull proportional hazards setting with covariate imbalance and effect modification. Weibull PH had the lowest LOOIC in both ML-NMR and IPD NMA, ML-NMR recovered true parameters closely, and standard errors for unobserved comparisons were modestly higher by about $W$8–$W$9. In the multiple myeloma application, population-average survival curves fit observed Kaplan-Meier curves well, proportional hazards were favored overall under ML-NMR, and the method addressed limitations of MAIC-then-NMA by giving a coherent synthesis across all studies and estimates for arbitrary target populations. The paper’s central conclusion is that the plug-in AVR approximation is biased whenever nonlinear links, non-collapsible effects, or treatment-covariate interactions are present, whereas ML-NMR avoids this aggregation bias by integrating the individual likelihood over the aggregate-data covariate distribution (Phillippo et al., 2024).

6. Clustering-based AVR for aggregate forecasting

In "Clustering-based aggregate value regression," AVR is defined for forecasting aggregate values in a linear regression model. There are $f(\cdot;W)$0 units, each with response vector $f(\cdot;W)$1 and predictors $f(\cdot;W)$2, and the aggregate response is

$f(\cdot;W)$3

Each unit satisfies

$f(\cdot;W)$4

with $f(\cdot;W)$5 and $f(\cdot;W)$6. AVR combines all regression models into a single model by setting

$f(\cdot;W)$7

so that

$f(\cdot;W)$8

Training minimizes the aggregate squared error

$f(\cdot;W)$9

If $\theta$0 is invertible, the estimator is ordinary least squares,

$\theta$1

If the system is rank-deficient, the estimator is the Moore-Penrose minimum-norm least squares solution $\theta$2, described as ridgeless least squares.

Because AVR has $\theta$3 parameters, it can be overparameterized when $\theta$4 is large. The proposed remedy is AVR-C, which partitions the $\theta$5 models into $\theta$6 disjoint clusters $\theta$7. For a cluster $\theta$8,

$\theta$9

and the cluster model is

$$Z = f(X;W) + \epsilon,\qquad \epsilon \sim \mathcal{N}(0,\sigma^2).$$0

AVR is then fit independently within each cluster. IR corresponds to $$Z = f(X;W) + \epsilon,\qquad \epsilon \sim \mathcal{N}(0,\sigma^2).$$1, AVR to $$Z = f(X;W) + \epsilon,\qquad \epsilon \sim \mathcal{N}(0,\sigma^2).$$2, and AVR-C interpolates between them, with $$Z = f(X;W) + \epsilon,\qquad \epsilon \sim \mathcal{N}(0,\sigma^2).$$3 controlling model complexity.

Two clustering procedures are proposed. Training Error Minimization (TEM) is a hierarchical agglomerative method that starts from singleton clusters and repeatedly merges the pair yielding the smallest total training RSS; its computational complexity is $$Z = f(X;W) + \epsilon,\qquad \epsilon \sim \mathcal{N}(0,\sigma^2).$$4 least-squares fits along the full path. Residual Correlation Matrix (RCM) clustering first fits individual regressions, computes residual correlations $$Z = f(X;W) + \epsilon,\qquad \epsilon \sim \mathcal{N}(0,\sigma^2).$$5, defines dissimilarities $$Z = f(X;W) + \epsilon,\qquad \epsilon \sim \mathcal{N}(0,\sigma^2).$$6, and applies standard hierarchical clustering such as Ward’s method. TEM is more accurate but computationally heavier; RCM is faster and more interpretable.

The theoretical analysis assumes a misspecified true model

$$Z = f(X;W) + \epsilon,\qquad \epsilon \sim \mathcal{N}(0,\sigma^2).$$7

where $$Z = f(X;W) + \epsilon,\qquad \epsilon \sim \mathcal{N}(0,\sigma^2).$$8 are latent predictors omitted from the working model. Under positive cross-cluster error covariances, the expected training error is monotone non-increasing as clusters are merged, so training RSS decreases as $$Z = f(X;W) + \epsilon,\qquad \epsilon \sim \mathcal{N}(0,\sigma^2).$$9 decreases. For test error, the difference between AVR and IR decomposes into nonnegative variance terms and nonpositive bias-reduction terms. Under correct specification, merging increases test error, so IR is best. Under misspecification, aggregation can reduce bias enough to dominate the variance increase, yielding a U-shaped test error curve as a function of $x_i \in X$00. The paper explicitly connects this behavior to bias-variance trade-off theory under misspecification, and also reports double descent phenomena in high-dimensional regimes.

The Monte Carlo simulations use $x_i \in X$01 with $x_i \in X$02 and $x_i \in X$03 entries i.i.d. $x_i \in X$04, coefficients i.i.d. $x_i \in X$05, training sizes $x_i \in X$06, test size $x_i \in X$07, $x_i \in X$08, $x_i \in X$09, and $x_i \in X$10 or $x_i \in X$11. For $x_i \in X$12, training error increases monotonically with $x_i \in X$13, and test error is clearly U-shaped with an optimal $x_i \in X$14 near $x_i \in X$15, close to the true error-correlation cluster count. For $x_i \in X$16, similar behavior is observed for moderate $x_i \in X$17, but double descent appears when $x_i \in X$18 is small and interpolation occurs within clusters.

The electricity-demand case study uses hourly electricity consumption from $x_i \in X$19 facilities in Japan, selecting $x_i \in X$20 facilities with complete data from Jan 5 to Dec 31, 2014. Training uses the first $x_i \in X$21 days and testing the last $x_i \in X$22 days. Predictors include past demand with autoregressive lag $x_i \in X$23 days, daily mean temperature for $x_i \in X$24 areas, and weekday/holiday dummies. IR has $x_i \in X$25 parameters per facility, whereas AVR has $x_i \in X$26 parameters overall, exceeding the $x_i \in X$27 training observations. In the Chugoku area ($x_i \in X$28), training RMSE is nearly monotone increasing in $x_i \in X$29, test RMSE is U-shaped, and minima occur at $x_i \in X$30 for TEM and $x_i \in X$31 for RCM. For all areas ($x_i \in X$32), training RMSE again increases with $x_i \in X$33, while test RMSE exhibits double descent. The reported practical recommendation is to select $x_i \in X$34 on the first-descent side where test RMSE is minimized; both clustering methods outperform pure IR and avoid the overfitting of global AVR at moderate $x_i \in X$35 (Hirose et al., 21 Aug 2025).