---
title: 'Hyperloss: Learnable Loss & Quantum Regimes'
url: https://www.emergentmind.com/topics/hyperloss
type: topic
---

# Hyperloss: Learnable Loss & Quantum Regimes

to=arxiv_search.search  微信天天彩票json
{"query":"hyperloss loss learning 2007.06059 2002.03555 2603.21982", "max_results": 10}
to=arxiv_search.search  久赢json
{"query":"all:hyperloss OR ti:\"It Is Likely That Your Loss Should be a Likelihood\" OR ti:\"Supervised Learning: No Loss No Cry\" OR ti:\"Hyperloss from coherent spatial-mode mixing in quantum-correlated networks\"", "max_results": 10}
to=arxiv_search.search ＿色json
{"query":"ti:\"It Is Likely That Your Loss Should be a Likelihood\"", "max_results": 5}
Hyperloss denotes a class of constructions in which the loss is itself treated as an object of optimization rather than as a fixed design choice. In machine learning, this includes learning likelihood parameters such as variances, temperatures, shapes, and prior scales; learning proper composite losses through canonical links, Bregman divergences, or source functions; generating symbolic losses automatically; and building hybrid task-specific objectives. In quantum optics, the same term denotes a distinct physical regime in which coherent spatial-mode mixing makes the apparent loss of squeezing exceed the limit implied by ordinary beam-splitter loss, even though the underlying multimode evolution remains coherent [2007.06059; 2603.21982].

## 1. Learned likelihoods and trainable loss parameters

A probabilistic formulation of hyperloss begins from the claim that many standard objectives are special cases of negative log-likelihoods with hard-coded distributional parameters. Mean-squared error is the negative log-likelihood of a Normal distribution with fixed variance; cross-entropy is the negative log-likelihood of a softmax model with fixed temperature; and \(L_2\) and \(L_1\) regularization correspond to Normal and Laplace priors with fixed scales. The proposed alternative is to optimize the full likelihood
\[
\min_{\theta,\phi}\;\mathcal{L}(\theta,\phi)=-\sum_i \log p(y_i\mid x_i,\theta,\phi),
\]
or, with adaptive priors,
\[
\mathcal{L}(\theta,\phi)
=
-\sum_i \log p(y_i \mid x_i, \theta, \phi_{\text{lik}})
-\log p(\theta \mid \phi_{\text{prior}}),
\]
where \(\theta\) are ordinary model parameters and \(\phi\) are learnable likelihood parameters controlling loss scale, shape, temperature, and regularization strength [2007.06059].

For scalar regression under a Normal model,
\[
\ell_N(\hat{y}, y; \sigma)
=
\frac{1}{2\sigma^2}(\hat{y}-y)^2 + \log \sigma.
\]
Fixing \(\sigma=1\) recovers standard MSE up to constants, while learning \(\sigma\) turns squared error into a family of losses indexed by scale. For classification with logits \(z\) and temperature \(\tau\),
\[
p(\hat{y}=y\mid z,\tau)
=
\frac{\exp(z_y \tau)}{\sum_c \exp(z_c \tau)},
\]
so \(\tau=1\) recovers ordinary softmax cross-entropy, larger \(\tau\) yields a sharper softmax, and smaller \(\tau\) yields a flatter distribution. The same logic extends to robust regression through Barron’s generalized robust loss with learnable shape \(\alpha\) and scale \(\sigma\), and to adaptive regularization through learned prior scales \(\sigma_w\), \(b\), or element-wise \(\sigma_j\), \(b_j\).

This formulation treats loss parameters like ordinary trainable variables. Positive parameters such as \(\sigma\) and \(\tau\) are obtained through softplus or shifted softplus; bounded parameters such as \(\alpha \in [0,3]\) are obtained through an affine sigmoid. Conditioning can be global, data-indexed, or predicted from inputs or features, and dimensionality can be scalar, per-channel, or per-pixel. In this sense, hyperloss is a hyperparameterized loss whose scale and shape are optimized jointly with the predictor. The reported applications include robust modeling, outlier detection, recalibration, and adaptive regularization: on the ODDS benchmark with 22 datasets, PCA+S and AE+S attained median AUC 0.868 and 0.846; for classification recalibration, Linear Scaling and Linear Feature Scaling often reduced ECE more than Global Scaling, Platt, Isotonic, or Vector Scaling; and for robust VAE reconstruction on CelebA, predicted per-pixel \(\alpha(x)\) and \(\sigma(x)\) yielded the best reconstruction MSE among the compared variants [2007.06059].

## 2. Proper losses, Bregman structure, and Bayesian loss learning

A second, more explicitly meta-level meaning of hyperloss is learning the loss function itself within a structured family. In binary supervised learning, this viewpoint appears in work that reinterprets SLIsotron as learning a composite square loss and then generalizes it to proper losses characterized by Bregman divergences. The core statement is that, for differentiable proper losses with invertible link \(\psi\), a proper composite loss is exactly a Bregman divergence:
\[
\ell(y^*, h(\mathbf{x}))
=
D_{-\cbr}\big(y \,\Vert\, \psi^{-1}\circ h(\mathbf{x})\big),
\]
with an equivalent dual form in linked space. Learning the canonical link is therefore equivalent to learning the loss, and BregmanTron operationalizes this by jointly updating the classifier and the inverse link \(u\), which determines the convex primitive \(U\), the conditional Bayes risk \(\cbr_u\), and the proper canonical loss induced by \(u\) [2002.03555].

BregmanTron treats the hyperloss family as the set of proper canonical losses indexed by admissible inverse links \(u:\mathbb{R}\to[0,1]\) satisfying monotonicity and slope constraints. The algorithm alternates between a classifier update and a Bregman-regularized prediction update under monotonicity and Lipschitz constraints, followed by piecewise linear reconstruction of the next link. Theoretical results include a monotone decrease of the learned loss under the stated \(\delta\)-regime and stability conditions, and an agnostic approximability result showing that piecewise affine links can approximate any target canonical link under the paper’s metric. Empirically, on the reported binary tasks, CLU matched or nearly matched logistic regression and substantially outperformed SLIsotron: on the synth task, Logistic regression obtained 92.2\%, SLIsotron 91.6\%, and CLU 92.3\%; on mnist, Logistic 99.9\%, SLIsotron 94.6\%, CLU 99.7\%; on fmnist, Logistic 98.5\%, SLIsotron 90.7\%, CLU 97.9\%. The same work also demonstrated loss transfer: a loss learned on Fashion-MNIST task \(0\) vs \(6\) reached AUC 0.879 when transferred to task \(2\) vs \(4\), matching CLU trained directly on the target task and slightly exceeding logistic regression at 0.877 [2002.03555].

A related Bayesian line replaces direct link learning with learning a source function
\[
\nu = -\cbr' \circ \chi^{-1},
\]
where \(\chi\) is the composite link. The source is modeled by an Integrated Squared Gaussian Process,
\[
\nu(x)=\nu_0+\int_0^x f^2(z)\,dz,
\]
which yields a differentiable, strictly increasing function with probability one. This construction preserves properness-related structure while making Bayesian inference over losses feasible. With a universal kernel, the support of the prior can represent any proper composite loss; with \(\mu=0\) and \(k(0,0)=1\), the prior mean is \(\nu(x)=x\), corresponding to the canonical link of log-loss in the GLM setting. The resulting ISGP-Linkgistic model was reported to match or outperform logistic regression across MNIST, Fashion-MNIST, and KMNIST, and to be the first loss-learning method in the reported experiments that significantly beat logistic regression on Fashion-MNIST [2006.04633].

## 3. Loss-oblivious prediction and automatic loss generation

Hyperloss can also denote a framework in which training is performed without committing to a single downstream loss. An \((\mathcal{L},\mathcal{C},\delta)\)-omnipredictor is a predictor \(f:X\to[0,1]\) such that for every \(\ell\in\mathcal{L}\) there exists a univariate post-processing map \(k_\ell\) with
\[
\ell_D(k_\ell\circ f)
\le
\min_{c\in\mathcal{C}} \ell_D(c) + \delta.
\]
The loss family considered consists of convex, Lipschitz, \((B,\varepsilon)\)-nice losses, and the construction is obtained through multicalibrated partitions. The key theorem states that if a partition is \(\alpha\)-approximately multicalibrated for \(\mathcal{C}\), then its canonical predictor is an \((\mathcal{L},\mathcal{C},2\alpha B+\varepsilon)\)-omnipredictor. In this formulation, hyperloss is not a single learnable loss but a guarantee that one predictor can, after simple post-processing, serve a family of losses and even multi-group loss minimization tasks [2109.05389].

A different extreme is automatic symbolic loss synthesis. AutoLossGen defines a search space of losses built from the primitive operands \(\{\hat{y}, y, 1\}\) and the operators Add, Multi, Max, Min, Neg, Identical, Log, Square, and Reciprocal. A two-layer LSTM controller samples loss expressions, and reinforcement learning updates the controller from validation reward. The framework uses a proxy test with a zero-gradient filter and a gradient-duplicate filter, together with reward filtering that accepts model updates only when the reward is not too negative. The paper reports that the proxy test increased the number of sampled losses in 24 hours from about 3,200 to about 93,792, a speed-up of about \(30\times\). It also reports several generated losses, including
\[
\mathcal{L}_{\text{MaxR}}(\hat{y}, y)
=
\max\left(\frac{\hat{y} + \epsilon}{y + \epsilon}, \frac{y + \epsilon}{\hat{y} + \epsilon}\right),
\]
\[
\mathcal{L}_{\text{SumR}}(\hat{y}, y)
=
\frac{\hat{y} + y + \epsilon}{\hat{y}\cdot y + \epsilon},
\]
and
\[
\mathcal{L}_{\text{LogMin}}(\hat{y}, y)
=
\log\left(\frac{1 + \epsilon}{\min(\hat{y}, y) + \epsilon}\right)\cdot \big(\hat{y} + y + \min(\hat{y}, y)\big).
\]
On MF–ML-100K classification, the paper reports AUC 0.8087 for MaxR versus 0.7930 for the best listed baseline, Focal; on MLP–Electronics classification, it reports AUC 0.6404 for SumR. For regression, AutoLossGen repeatedly rediscovered MSE [2204.13160].

## 4. Task-specific hybrid objectives

In application-specific settings, hyperloss may take the form of a hybrid loss built to encode multiple error asymmetries and evaluation desiderata at once. HyTver, proposed for longitudinal multiple sclerosis lesion segmentation, combines a Tversky-index loss with a modified class-weighted binary cross-entropy:
\[
L_{HT} = \gamma L_{mCE} + (1-\gamma)L_{TI}.
\]
The Tversky component is
\[
TI(y,\hat{y})
=
\frac{\sum_i y_i \hat{y}_i}
{\sum_i y_i \hat{y}_i
+
\alpha \sum_i (1-\hat{y}_i)y_i
+
(1-\alpha)\sum_i \hat{y}_i(1-y_i)},
\qquad
L_{TI}=1-TI(y,\hat{y}),
\]
and the modified cross-entropy component is
\[
L_{mCE}
=
-\frac{1}{N}\sum_i
\left[
\beta y_i \log(p_i)
+
(1-\beta)(1-y_i)\log(1-p_i)
\right].
\]
Here \(\alpha\) controls the false-negative versus false-positive weighting in Tversky, \(\beta\) controls the same trade-off in the voxel-wise term, and \(\gamma\) balances region-level and voxel-level objectives [2508.17639].

The motivation is explicitly longitudinal: lesions are tiny relative to background, baseline lesions must not be relabeled as new lesions, and both input imbalance and output imbalance are severe. HyTver is presented as computationally simpler than exponent-based alternatives while preserving complementary region and boundary behavior. On the reported test set, the paper gives the following values for HyTver: Dice coefficient 0.659, Jaccard index 0.524, Hausdorff distance 36.4, Average Surface Distance 6.83, Precision 0.724, and F1 0.747. In the same discussion, the paper states that HyTver had the lowest coefficient of variation for Dice and Precision among the tested losses and a relatively low coefficient of variation for F1 in the pre-trained setting. Within the paper’s terminology, this is a concrete task-specific hyperloss: a single scalar objective that explicitly combines overlap control, FP/FN asymmetry, and voxel-wise discrimination [2508.17639].

## 5. Hyperloss in quantum-correlated networks

In quantum optics, hyperloss denotes a specific decoherence regime rather than a meta-learning technique. Ordinary optical loss for a squeezed state is modeled as a beam splitter mixing the signal with vacuum:
\[
\Delta^2 \hat{X}_{\text{meas}}(\Omega)
=
(1-\lambda_{\rm bs})\,\Delta^2 \hat{X}_{\text{in}}(\Omega)
+
\lambda_{\rm bs}\,\Delta^2 \hat{X}_{\text{vac}}(\Omega).
\]
Within this model, the apparent loss cannot exceed complete replacement by vacuum. The hyperloss paper shows that this picture fails when a squeezed fundamental mode mixes coherently with higher-order spatial modes. In the hot spatial-mode-mixing regime, the measured squeezed-quadrature variance is
\[
\Delta^2 \hat{X}_{\text{meas}}(\Omega)
=
(1-\lambda_{\rm smm})\,\Delta^2 \hat{X}_{\text{in}}(\Omega)
+
\lambda_{\rm smm}\,\Delta^2 \hat{X}_{\text{vac}}(\Omega)
+
T(\Omega),
\]
where \(T(\Omega)\) is an additional thermal term arising from anti-squeezed fluctuations coherently mixed back into the measured quadrature. In the weak-coupling limit,
\[
\lambda_{\rm smm}
=
k_1^2 + k_2^2 + 2k_1k_2\cos\phi_1,
\qquad
T
=
k_1^2k_2^2 e^{2r_s}\sin^2\phi_1.
\]
The regime is called hyperloss when the apparent loss of squeezing, if interpreted through a pure beam-splitter model, exceeds \(100\%\) of the initial squeezing resource [2603.21982].

The mechanism depends on coherent mode mixing and phase accumulation. The paper distinguishes cold SMM, where coherent-state power redistribution remains bounded by \(100\%\), from hot SMM, where a pure quantum-correlated input causes the traced-over fundamental mode to appear thermal-like because anti-squeezed fluctuations in higher-order modes contaminate the measured quadrature. Three special phases are highlighted for equal mismatch \(k_1=k_2=k\): \(\phi_1=\pi\) is the recovery phase, \(\phi_1=0\) is the cold maximum-loss phase, and \(\phi_1=\pi/2\) is the hyperloss phase. In the latter case,
\[
S_{y,\text{meas}}\big(\phi_1=\tfrac{\pi}{2},k\big)
=
e^{-2r_s}\cos^4 k
+
2\cos^2 k \sin^2 k
+
e^{2r_s}\sin^4 k,
\]
which reduces in the stated limit to \(k^4 e^{2r_s}\).

The experimental realization used two sequential, strongly overcoupled Fabry–Perot cavities with \(\mathrm{LG}_{00}\) as the fundamental mode and \(\mathrm{LG}_{01}\) as the higher-order mode. The reported squeezed-light source delivered up to \(5.8\pm0.5\,\text{dB}\) of measured squeezing and \(24\pm0.2\,\text{dB}\) of anti-squeezing at \(3.75\) MHz in a reference configuration with about \(26\%\) total optical loss. With only \(8\%\) mode mismatch, the paper reports that a \(5.8\,\text{dB}\) squeezed state was converted into an effectively thermal state with no quadrature squeezing; the minimal quadrature variance became larger than the shot-noise level by about \(1.5\,\text{dB}\), while the anti-squeezed quadrature remained about \(22\,\text{dB}\) above shot noise. Because the process is coherent, the lost correlations can be recovered by tuning the FM–HOM phase: the same work reports recovery of up to \(5.2\,\text{dB}\) of the initial \(5.8\,\text{dB}\) with about \(15\%\) geometric mismatch, corresponding to only about \(2.8\%\) effective loss on squeezing [2603.21982].

## 6. Relations, distinctions, and recurring misconceptions

The cited literature uses “hyperloss” in multiple technically distinct ways. In machine learning, it can mean trainable likelihood parameters inside a fixed probabilistic family, direct learning of a proper or composite loss, symbolic loss generation by reinforcement learning, a task-specific hybrid objective, or a loss-oblivious predictor that supports many downstream losses after post-processing. In quantum optics, it names a regime of apparent squeezing degradation caused by coherent spatial-mode mixing rather than a learned objective. This suggests that the term has scope rather than a single canonical definition.

Several misconceptions are explicitly addressed by the underlying works. In the likelihood-based machine-learning formulation, learning the loss does not mean arbitrary removal of regularization or loss scaling: the normalization terms such as \(\log\sigma\), \(\log\tau\), and \(\log b\) prevent trivial solutions and tie the objective to a probabilistic model [2007.06059]. In the proper-loss literature, loss learning is not unconstrained search over arbitrary functions; the admissible families are restricted by properness, Bregman structure, canonical links, monotonicity, or source-function priors, which preserve Bayes alignment and calibration properties [2002.03555; 2006.04633]. In the quantum-optical literature, hyperloss is not literal optical power loss greater than \(100\%\); it is an apparent excess loss of squeezing obtained when coherent anti-squeezed fluctuations from unmeasured modes are misinterpreted through a scalar beam-splitter-loss model [2603.21982].

A further recurring theme is transferability. BregmanTron reports loss transfer between related Fashion-MNIST tasks, and AutoLossGen reports that most generated losses are transferable across models or datasets [2002.03555; 2204.13160]. By contrast, HyTver is explicitly task-specific, with its design tied to longitudinal lesion appearance, small foreground structures, and asymmetric clinical costs [2508.17639]. The resulting picture is technically heterogeneous but conceptually coherent: hyperloss names situations in which the structure that determines how errors are penalized is elevated to a learnable, controllable, or physically dynamical object, rather than treated as a fixed scalar formula.

Source: https://www.emergentmind.com/topics/hyperloss