---
title: Relative Neural Efficiency in Neuroimaging & ML
url: https://www.emergentmind.com/topics/relative-neural-efficiency
type: topic
---

# Relative Neural Efficiency in Neuroimaging & ML

Searching arXiv for the cited papers to ground the article.
Searching for neuroimaging efficiency/capacity paper.
Searching for training-efficiency framework paper.
Searching for inference efficiency scoring paper.
Relative neural efficiency denotes a family of comparative constructs used to relate neural response or neural-system performance to some measure of demand, cost, or resource consumption. In task-based neuroimaging, the central quantities are the rate at which BOLD activity increases with cognitive load and the upper limit of that increase, yielding efficiency and capacity parameters in a quadratic activity–load model [1606.06249]. In recent machine-learning work, the term is used more explicitly for comparative training efficiency, defined as the ratio between architecture-level efficiency scores derived from accuracy and logged power consumption during training [2409.07925]. A related inference-time literature uses a measurement-based composite score, $S=\frac{\text{Accuracy}^2}{E_{\text{inf}}}$, to compare architectures and hardware by energy per inference rather than training cost [2310.09554]. These usages are methodologically related in that they replace absolute performance with comparative efficiency judgments, but they are not reducible to a single universal metric.

## 1. Domain-specific meanings

The expression has different technical meanings across the cited literature. In Steffener et al., neural efficiency and neural capacity are formal constructs, but “relative neural efficiency” is not introduced as a formal term; instead, neural efficiency is operationalized from the slope parameter of a quadratic BOLD–load function [1606.06249]. In the training-efficiency framework of 2024, relative neural efficiency is the comparative ability of different neural architectures to convert training resources, measured from hardware power logs, into task performance, and is formalized as a ratio between architecture-level efficiency scores [2409.07925]. In the inference-efficiency scoring work of 2023, relative comparison is performed through a composite score that trades off accuracy against measured energy per inference, with cross-model or cross-hardware comparison often expressed as a simple ratio to a baseline system [2310.09554].

| Domain | Base efficiency quantity | Relative comparison |
|---|---|---|
| Task fMRI | Slope and peak of a quadratic activity–load curve | Normalization of slope by curvature, peak, or baseline is a grounded extension |
| Training of neural architectures | $\text{Accuracy} / \sum W_n$ accumulated over training | $R_{A,B} = \mathrm{Eff}_A / \mathrm{Eff}_B$ |
| Inference on hardware | $S=\frac{\text{Accuracy}^2}{E_{\text{inf}}}$ | Ratio across models or relative to baseline hardware |

A separate but related strand is relativistic architecture evaluation in NAS. ReNAS argues that the relevant problem is to determine which architecture would perform better rather than accurately predict absolute architecture performance [1910.01523]. That formulation concerns relative ranking, not neural efficiency per se, but it is conceptually adjacent because it prioritizes pairwise or comparative evaluation over absolute-score estimation.

## 2. Neuroimaging formulation: efficiency, capacity, and differential equations

In the neuroimaging setting, brain activity measured with fMRI BOLD during a working-memory n-back task is modeled as a function of cognitive load coded as $t \in \{0,1,2,3\}$ [1606.06249]. Neural efficiency is the rate at which brain activity increases with cognitive load, while neural capacity is the maximal brain activity reached as cognitive load increases. The paper’s central reinterpretation is that the standard quadratic model is the solution of a first-order differential equation:

$$
\frac{dx}{dt} = 2At + B
$$

with constant second derivative

$$
\frac{d^2x}{dt^2} = 2A,
$$

and quadratic solution

$$
x(t)=At^2+Bt+C.
$$

Here, $x(t)$ is brain activity in percent signal change, $B$ is the “velocity” or baseline slope, $2A$ is the constant “acceleration” or curvature, and $C$ is the intercept or baseline activity at $t=0$ [1606.06249].

The sign of $A$ governs the capacity regime. When $A<0$, the curve is concave downward and exhibits an upper bound with an inverted-U peak at a finite load. When $A=0$, the behavior is strictly linear. When $A>0$, curvature is upward-opening and capacity is unconstrained in the studied range. The paper maps these regimes onto three classifications: capacity dependent, capacity independent, and unconstrained by capacity. Capacity dependence occurs when activity rises and then falls within the studied range; capacity independence occurs when the peak lies beyond the highest observed load; unconstrained behavior corresponds to upward curvature or continued increase without an interpretable peak in the range [1606.06249].

For the capacity-limited case, cognitive capacity and neural capacity are derived directly from the quadratic coefficients:

$$
t^* = \mathrm{Cogcap} = -\frac{B}{2A}
$$

and

$$
\mathrm{Ncap} = x(t^*) = -\frac{B^2}{4A}+C.
$$

The paper’s operational neural efficiency is not the slope at $t=0$, which equals $B$, but the slope midway between the lowest load and the cognitive capacity. Evaluated at $t=t^*/2$, this gives

$$
\mathrm{Neff} = \frac{B}{2},
$$

a definition used in Section 2.6 and Figure 2 of the paper [1606.06249].

## 3. Estimation in the n-back GLM and grounded relative extensions

The neuroimaging implementation uses four n-back levels in a fixed order $(0,1,2,3,0,1,2,3)$, with each block lasting 32 s, alternating with 24 s fixation, and $\mathrm{TR}=2.5$ s [1606.06249]. The first-level GLM models each load as a separate block regressor convolved with the canonical HRF in SPM12. To standardize units, the paper converts load-specific beta estimates to percent signal change using

$$
\mathrm{PSC}\,\beta_i = 100 \times \frac{\beta_i}{\beta_5}, \qquad i=1,\dots,4,
$$

where $\beta_5$ is the estimated mean signal regressor [1606.06249].

Orthogonal polynomial contrasts are then used to recover the quadratic parameters voxelwise. Using PSC beta maps for 0-, 1-, 2-, and 3-back in order, the supplied weights are:

- $C$: $[0.95,\,0.15,\,-0.15,\,0.05]$
- $B$: $[-1.05,\,0.65,\,0.85,\,-0.45]$
- $A$: $[0.25,\,-0.25,\,-0.25,\,0.25]$

These directly parameterize the derivative structure, since $\frac{dx}{dt}=2At+B$ and $\frac{d^2x}{dt^2}=2A$ [1606.06249]. The same framework also defines AUC and scaled AUC through linear contrasts, with
$$
\mathrm{AUC} = 9A + 4.5B + 3C.
$$

Empirically, the AUC contrast revealed robust load-related activation across bilateral inferior, middle, and superior frontal gyri, supplementary motor area, cingulate, insula, posterior parietal, and occipital cortex, and percent signal change above baseline with increasing load ranged approximately $0.1\%$–$0.35\%$ [1606.06249]. The right insula is given as a capacity-dependent example with $A=-0.058$, $B=0.194$, $C=0.057$, $\mathrm{Neff}=0.097$, $\mathrm{Cogcap}=1.684$, and $\mathrm{Ncap}=0.221$. The SMA is given as a capacity-independent example with $A=-0.008$, $B=0.097$, $C=0.206$, $\mathrm{Neff}=0.048$, $\mathrm{Cogcap}=6.471$, and $\mathrm{Ncap}=0.520$ [1606.06249].

Because “relative neural efficiency” is not a formal construct in the paper, the supplied interpretation proposes grounded relative metrics consistent with the framework. These include normalization of slope by curvature,
$$
E_{\mathrm{rel1}} = \frac{B}{|A|},
$$
normalization of operational efficiency by peak activity,
$$
E_{\mathrm{rel2}} = \frac{(B/2)}{\mathrm{Ncap}} = \frac{(B/2)}{-B^2/(4A)+C},
$$
and baseline-normalized slope,
$$
E_{\mathrm{rel3}} = \frac{B}{C}.
$$
This suggests a way to compare regions or participants when curvature, baseline activity, or peak amplitude differ [1606.06249].

## 4. Relative training efficiency of neural architectures

In the 2024 framework, training efficiency is defined as performance divided by resource usage, instantiated as accuracy over accumulated logged power during training [2409.07925]. The per-epoch cumulative training efficiency is

$$
E(\mathrm{Acc},W,i=\mathrm{epoch}) = \frac{\mathrm{Acc}_i}{\sum_{n=0}^{i}[W_n]},
$$

where $\mathrm{Acc}_i$ is the accuracy after epoch $i$, and $W_n$ is the sampled energy consumption rate constructed from GPU, CPU, and RAM readings. Although the paper refers to “total energy consumed measured in terms of Joules per second (Watts)” and uses an energy budget in “kW” as a stopping criterion, the framework consistently sums sampled power values and reports efficiency in “accuracy per Watt” [2409.07925].

To control for hyperparameter effects, the paper averages efficiency across five width-scaled size variants, yielding an architecture-level efficiency $\mathrm{Eff}(\mathrm{arch},j=\mathrm{size})$, and then averages across four stopping criteria, yielding $\mathrm{Eff}(\mathrm{arch},k=\mathrm{convergence})$ [2409.07925]. Relative neural efficiency between two architectures is then the ratio

$$
R_{A,B} = \frac{\mathrm{Eff}_A}{\mathrm{Eff}_B}.
$$

The main demonstration compares LeNet-5 CNNs and Bayesian counterparts on MNIST and CIFAR-10, with shared hyperparameters including Adam, learning rate $0.001$, batch size $256$, softplus activation, cross-entropy loss, and width scaling by filter multipliers $\times 1$ to $\times 5$ [2409.07925]. Training is evaluated under four stopping criteria: fixed epochs, accuracy threshold, early stopping with patience $p=3$, and an energy-bound criterion with accumulated “Energy==100kW” checked per epoch [2409.07925].

The principal findings are explicitly comparative. Global architecture-level efficiencies averaged over sizes and stopping criteria are reported as $12.86\times 10^{-6}$ for LeNet and $2.88\times 10^{-6}$ for BCNN on MNIST, giving $R_{\mathrm{LeNet},\mathrm{BCNN}}=4.46$, and $7.06\times 10^{-6}$ versus $1.36\times 10^{-6}$ on CIFAR-10, giving $R_{\mathrm{LeNet},\mathrm{BCNN}}=5.18$ [2409.07925]. The conclusion drawn in the paper is that CNNs are more efficient than BCNNs on both datasets and that the relative difference becomes more pronounced as the learning task becomes more complex [2409.07925].

Efficiency also decays monotonically as training progresses. On MNIST, epoch-wise efficiency drops more steeply and early-epoch maxima exceed $0.02$; on CIFAR-10, decay is slower and the maximum is below $0.01$ [2409.07925]. The paper attributes this to diminishing returns once performance plateaus. It further reports non-linear interactions among stopping criteria, model size, and efficiency. For example, on MNIST, LeNet’s mean efficiency spans from $8.09\times 10^{-6}$ under 50 epochs to $26.10\times 10^{-6}$ under the accuracy-bound criterion, whereas BCNN spans from $1.00\times 10^{-6}$ under early stopping to $6.18\times 10^{-6}$ under the energy-bound criterion [2409.07925].

## 5. Inference-time relative efficiency and hardware fit

The 2023 efficient-computing work moves from training to inference and introduces a measurement-based composite score that trades off accuracy against measured energy per inference [2310.09554]. The primary definition is

$$
S = \frac{\text{Accuracy}^2}{\text{Power consumption per inference}},
$$

with the paper clarifying that “power consumption per inference (in mWh)” is energy per inference. In expanded form,

$$
S = \frac{\left(\frac{\text{Number of correct inferences}}{\text{Number of inferences}}\right)^2}{\left(\frac{\text{Total Power consumption}}{\text{Number of inferences}}\right)},
$$

and in the paper’s “physical-sense simplification” this becomes

$$
S = \frac{\text{Accuracy}}{\text{Average power consumed to obtain one correct inference}}.
$$

Operationally, accuracy is measured on the ILSVRC2012 validation set, while energy is computed from telemetry integrated over the inference window:
$$
E = \int_{t_0}^{t_1} P(t)\,dt
\quad\text{or}\quad
E \approx \sum_{k=1}^{K} P_k\,\Delta t,
$$
followed by
$$
E_{\mathrm{inf}}=\frac{E_{\mathrm{total}}}{N_{\mathrm{inferences}}}.
$$
The monitoring stack combines NVIDIA DCGM for GPU power, RAPL via Scaphandre for CPU energy on x86, Node Exporter for system metrics, and Prometheus/Grafana for storage and visualization [2310.09554].

The paper reports three full validation runs per model and hardware, plus stability tests with ten chained runs. On the A100 platform, MobileNetV3 Small achieves accuracy $68.1\%$, $E_{\mathrm{inf}}=9\,\mu\mathrm{Wh}$, and score $51.5$, while EfficientNetV2-L achieves $85.7\%$, $530\,\mu\mathrm{Wh}$, and score $1.4$ [2310.09554]. The score therefore makes explicit a relative efficiency judgment: a lower-accuracy model can be far more efficient than a higher-accuracy model when energy per inference differs by orders of magnitude. The same paper reports that MobileNetV3 Small remains best across all tested platforms and that some architectures, such as EfficientNetV2-L, are more strongly penalized off the A100, with scores dropping to $0.48$ on Quadro, $0.04$ on Bi-Xeon, and $0.03$ on i7 relative to $1.4$ on A100 [2310.09554].

The authors describe this as a “fit test for neural architectures on hardware architectures.” The comparative logic is straightforward: for a fixed model and dataset, accuracy is unchanged across hardware, so cross-hardware score differences reduce to measured energy efficiency. A plausible implication is that inference-time relative neural efficiency can serve as a hardware-selection criterion just as directly as a model-selection criterion [2310.09554].

## 6. Methodological issues, misconceptions, and limitations

A recurrent misconception is that relative neural efficiency denotes a single metric applicable across neuroscience and machine learning. The cited literature does not support that interpretation. In the neuroimaging paper, the formal quantities are $\mathrm{Neff}$, $\mathrm{Cogcap}$, and $\mathrm{Ncap}$, and “relative neural efficiency” is only a grounded extension of that framework rather than a defined term in the paper [1606.06249]. In the 2024 training paper, by contrast, relative efficiency is explicitly a ratio of architecture-level efficiency scores [2409.07925]. In the 2023 inference paper, the relative comparison is score-based and may be normalized to a baseline hardware system for presentation, but that normalization is not part of the score definition [2310.09554].

Several technical caveats are also domain-specific. In neuroimaging, the quadratic approximation is fitted across only four discrete load levels, and the differential-equation reinterpretation assumes smooth changes in slope with constant second derivative $2A$ [1606.06249]. The paper notes that sigmoidal models such as Weibull or logistic may capture richer inflections but require nonlinear fitting and more load points, and that the fixed block order may introduce sequence effects [1606.06249]. Moreover, $\mathrm{Neff}=B/2$ is defined operationally for capacity-limited cases with $A<0$.

In the training-efficiency framework, the cost term is a sum of sampled power values rather than a physically explicit Joule integral, and the paper itself notes that a more physically consistent energy measure could refine units and thresholds [2409.07925]. The framework is also sensitive to stopping criteria and can be confounded by overtraining. The 100-epoch analysis shows that fixed long runs can distort relative comparisons: on CIFAR-10, the efficiency ratio between LeNet and BCNN falls to $1.095$ under 100-epoch scoring, which the paper identifies as misleading because LeNet is overtrained while BCNN is comparatively less overtrained [2409.07925].

In the inference-efficiency scoring literature, the score is explicitly inference-only and does not cover training efficiency [2310.09554]. The monitoring stack is hardware-dependent: CPU energy relies on RAPL for x86, and the GPU measurements shown are for NVIDIA via DCGM. The paper also notes that TensorFlow pre-allocates GPU memory, inflating occupancy statistics without necessarily altering energy totals; interpretation should therefore focus on measured energy rather than raw resource occupancy [2310.09554].

Taken together, these limitations indicate that relative neural efficiency is best understood as a comparative methodology rather than a single invariant observable. In one setting it compares slopes and peaks of neural activation; in another it compares architectures by accuracy per logged training cost; in another it compares model–hardware pairs by accuracy per measured inference energy. The unifying principle is comparative conversion of demand or resources into a target outcome, but the observables, units, and inferential scope remain domain-specific [1606.06249] [2409.07925] [2310.09554].

Source: https://www.emergentmind.com/topics/relative-neural-efficiency