Relative Neural Efficiency in Neuroimaging & ML

• Relative neural efficiency is a set of comparative metrics assessing how neural systems or models convert demands and resources into performance outcomes.
• In neuroimaging, it is quantified via quadratic BOLD–load models to capture both the rate of activity increase and the capacity limits during cognitive tasks.
• In machine learning, it evaluates training and inference by comparing accuracy against energy consumption, enabling relative assessments of architectures and hardware.

Searching arXiv for the cited papers to ground the article. Searching for neuroimaging efficiency/capacity paper. Searching for training-efficiency framework paper. Searching for inference efficiency scoring paper. Relative neural efficiency denotes a family of comparative constructs used to relate neural response or neural-system performance to some measure of demand, cost, or resource consumption. In task-based neuroimaging, the central quantities are the rate at which BOLD activity increases with cognitive load and the upper limit of that increase, yielding efficiency and capacity parameters in a quadratic activity–load model (Steffener et al., 2016). In recent machine-learning work, the term is used more explicitly for comparative training efficiency, defined as the ratio between architecture-level efficiency scores derived from accuracy and logged power consumption during training (Cueto-Mendoza et al., 2024). A related inference-time literature uses a measurement-based composite score, $S=\frac{\text{Accuracy}^2}{E_{\text{inf}}}$, to compare architectures and hardware by energy per inference rather than training cost (Waltsburger et al., 2023). These usages are methodologically related in that they replace absolute performance with comparative efficiency judgments, but they are not reducible to a single universal metric.

1. Domain-specific meanings

The expression has different technical meanings across the cited literature. In Steffener et al., neural efficiency and neural capacity are formal constructs, but “relative neural efficiency” is not introduced as a formal term; instead, neural efficiency is operationalized from the slope parameter of a quadratic BOLD–load function (Steffener et al., 2016). In the training-efficiency framework of 2024, relative neural efficiency is the comparative ability of different neural architectures to convert training resources, measured from hardware power logs, into task performance, and is formalized as a ratio between architecture-level efficiency scores (Cueto-Mendoza et al., 2024). In the inference-efficiency scoring work of 2023, relative comparison is performed through a composite score that trades off accuracy against measured energy per inference, with cross-model or cross-hardware comparison often expressed as a simple ratio to a baseline system (Waltsburger et al., 2023).

Domain	Base efficiency quantity	Relative comparison
Task fMRI	Slope and peak of a quadratic activity–load curve	Normalization of slope by curvature, peak, or baseline is a grounded extension
Training of neural architectures	$\text{Accuracy} / \sum W_n$ accumulated over training	$R_{A,B} = \mathrm{Eff}_A / \mathrm{Eff}_B$
Inference on hardware	$S=\frac{\text{Accuracy}^2}{E_{\text{inf}}}$	Ratio across models or relative to baseline hardware

A separate but related strand is relativistic architecture evaluation in NAS. ReNAS argues that the relevant problem is to determine which architecture would perform better rather than accurately predict absolute architecture performance (Xu et al., 2019). That formulation concerns relative ranking, not neural efficiency per se, but it is conceptually adjacent because it prioritizes pairwise or comparative evaluation over absolute-score estimation.

2. Neuroimaging formulation: efficiency, capacity, and differential equations

In the neuroimaging setting, brain activity measured with fMRI BOLD during a working-memory n-back task is modeled as a function of cognitive load coded as $t \in \{0,1,2,3\}$ (Steffener et al., 2016). Neural efficiency is the rate at which brain activity increases with cognitive load, while neural capacity is the maximal brain activity reached as cognitive load increases. The paper’s central reinterpretation is that the standard quadratic model is the solution of a first-order differential equation:

$\frac{dx}{dt} = 2At + B$

with constant second derivative

$\frac{d^2x}{dt^2} = 2A,$

and quadratic solution

$x(t)=At^2+Bt+C.$

Here, $x(t)$ is brain activity in percent signal change, $B$ is the “velocity” or baseline slope, $\text{Accuracy} / \sum W_n$0 is the constant “acceleration” or curvature, and $\text{Accuracy} / \sum W_n$1 is the intercept or baseline activity at $\text{Accuracy} / \sum W_n$2 (Steffener et al., 2016).

The sign of $\text{Accuracy} / \sum W_n$3 governs the capacity regime. When $\text{Accuracy} / \sum W_n$4, the curve is concave downward and exhibits an upper bound with an inverted-U peak at a finite load. When $\text{Accuracy} / \sum W_n$5, the behavior is strictly linear. When $\text{Accuracy} / \sum W_n$6, curvature is upward-opening and capacity is unconstrained in the studied range. The paper maps these regimes onto three classifications: capacity dependent, capacity independent, and unconstrained by capacity. Capacity dependence occurs when activity rises and then falls within the studied range; capacity independence occurs when the peak lies beyond the highest observed load; unconstrained behavior corresponds to upward curvature or continued increase without an interpretable peak in the range (Steffener et al., 2016).

For the capacity-limited case, cognitive capacity and neural capacity are derived directly from the quadratic coefficients:

$\text{Accuracy} / \sum W_n$7

and

$\text{Accuracy} / \sum W_n$8

The paper’s operational neural efficiency is not the slope at $\text{Accuracy} / \sum W_n$9, which equals $R_{A,B} = \mathrm{Eff}_A / \mathrm{Eff}_B$0, but the slope midway between the lowest load and the cognitive capacity. Evaluated at $R_{A,B} = \mathrm{Eff}_A / \mathrm{Eff}_B$1, this gives

$R_{A,B} = \mathrm{Eff}_A / \mathrm{Eff}_B$2

a definition used in Section 2.6 and Figure 1 of the paper (Steffener et al., 2016).

3. Estimation in the n-back GLM and grounded relative extensions

The neuroimaging implementation uses four n-back levels in a fixed order $R_{A,B} = \mathrm{Eff}_A / \mathrm{Eff}_B$3, with each block lasting 32 s, alternating with 24 s fixation, and $R_{A,B} = \mathrm{Eff}_A / \mathrm{Eff}_B$4 s (Steffener et al., 2016). The first-level GLM models each load as a separate block regressor convolved with the canonical HRF in SPM12. To standardize units, the paper converts load-specific beta estimates to percent signal change using

$R_{A,B} = \mathrm{Eff}_A / \mathrm{Eff}_B$5

where $R_{A,B} = \mathrm{Eff}_A / \mathrm{Eff}_B$6 is the estimated mean signal regressor (Steffener et al., 2016).

Orthogonal polynomial contrasts are then used to recover the quadratic parameters voxelwise. Using PSC beta maps for 0-, 1-, 2-, and 3-back in order, the supplied weights are:

• $R_{A,B} = \mathrm{Eff}_A / \mathrm{Eff}_B$7: $R_{A,B} = \mathrm{Eff}_A / \mathrm{Eff}_B$8
• $R_{A,B} = \mathrm{Eff}_A / \mathrm{Eff}_B$9: $S=\frac{\text{Accuracy}^2}{E_{\text{inf}}}$0
• $S=\frac{\text{Accuracy}^2}{E_{\text{inf}}}$1: $S=\frac{\text{Accuracy}^2}{E_{\text{inf}}}$2

These directly parameterize the derivative structure, since $S=\frac{\text{Accuracy}^2}{E_{\text{inf}}}$3 and $S=\frac{\text{Accuracy}^2}{E_{\text{inf}}}$4 (Steffener et al., 2016). The same framework also defines AUC and scaled AUC through linear contrasts, with

$S=\frac{\text{Accuracy}^2}{E_{\text{inf}}}$5

Empirically, the AUC contrast revealed robust load-related activation across bilateral inferior, middle, and superior frontal gyri, supplementary motor area, cingulate, insula, posterior parietal, and occipital cortex, and percent signal change above baseline with increasing load ranged approximately $S=\frac{\text{Accuracy}^2}{E_{\text{inf}}}$6–$S=\frac{\text{Accuracy}^2}{E_{\text{inf}}}$7 (Steffener et al., 2016). The right insula is given as a capacity-dependent example with $S=\frac{\text{Accuracy}^2}{E_{\text{inf}}}$8, $S=\frac{\text{Accuracy}^2}{E_{\text{inf}}}$9, $t \in \{0,1,2,3\}$0, $t \in \{0,1,2,3\}$1, $t \in \{0,1,2,3\}$2, and $t \in \{0,1,2,3\}$3. The SMA is given as a capacity-independent example with $t \in \{0,1,2,3\}$4, $t \in \{0,1,2,3\}$5, $t \in \{0,1,2,3\}$6, $t \in \{0,1,2,3\}$7, $t \in \{0,1,2,3\}$8, and $t \in \{0,1,2,3\}$9 (Steffener et al., 2016).

Because “relative neural efficiency” is not a formal construct in the paper, the supplied interpretation proposes grounded relative metrics consistent with the framework. These include normalization of slope by curvature,

$\frac{dx}{dt} = 2At + B$0

normalization of operational efficiency by peak activity,

$\frac{dx}{dt} = 2At + B$1

and baseline-normalized slope,

$\frac{dx}{dt} = 2At + B$2

This suggests a way to compare regions or participants when curvature, baseline activity, or peak amplitude differ (Steffener et al., 2016).

4. Relative training efficiency of neural architectures

In the 2024 framework, training efficiency is defined as performance divided by resource usage, instantiated as accuracy over accumulated logged power during training (Cueto-Mendoza et al., 2024). The per-epoch cumulative training efficiency is

$\frac{dx}{dt} = 2At + B$3

where $\frac{dx}{dt} = 2At + B$4 is the accuracy after epoch $\frac{dx}{dt} = 2At + B$5, and $\frac{dx}{dt} = 2At + B$6 is the sampled energy consumption rate constructed from GPU, CPU, and RAM readings. Although the paper refers to “total energy consumed measured in terms of Joules per second (Watts)” and uses an energy budget in “kW” as a stopping criterion, the framework consistently sums sampled power values and reports efficiency in “accuracy per Watt” (Cueto-Mendoza et al., 2024).

To control for hyperparameter effects, the paper averages efficiency across five width-scaled size variants, yielding an architecture-level efficiency $\frac{dx}{dt} = 2At + B$7, and then averages across four stopping criteria, yielding $\frac{dx}{dt} = 2At + B$8 (Cueto-Mendoza et al., 2024). Relative neural efficiency between two architectures is then the ratio

$\frac{dx}{dt} = 2At + B$9

The main demonstration compares LeNet-5 CNNs and Bayesian counterparts on MNIST and CIFAR-10, with shared hyperparameters including Adam, learning rate $\frac{d^2x}{dt^2} = 2A,$0, batch size $\frac{d^2x}{dt^2} = 2A,$1, softplus activation, cross-entropy loss, and width scaling by filter multipliers $\frac{d^2x}{dt^2} = 2A,$2 to $\frac{d^2x}{dt^2} = 2A,$3 (Cueto-Mendoza et al., 2024). Training is evaluated under four stopping criteria: fixed epochs, accuracy threshold, early stopping with patience $\frac{d^2x}{dt^2} = 2A,$4, and an energy-bound criterion with accumulated “Energy==100kW” checked per epoch (Cueto-Mendoza et al., 2024).

The principal findings are explicitly comparative. Global architecture-level efficiencies averaged over sizes and stopping criteria are reported as $\frac{d^2x}{dt^2} = 2A,$5 for LeNet and $\frac{d^2x}{dt^2} = 2A,$6 for BCNN on MNIST, giving $\frac{d^2x}{dt^2} = 2A,$7, and $\frac{d^2x}{dt^2} = 2A,$8 versus $\frac{d^2x}{dt^2} = 2A,$9 on CIFAR-10, giving $x(t)=At^2+Bt+C.$0 (Cueto-Mendoza et al., 2024). The conclusion drawn in the paper is that CNNs are more efficient than BCNNs on both datasets and that the relative difference becomes more pronounced as the learning task becomes more complex (Cueto-Mendoza et al., 2024).

Efficiency also decays monotonically as training progresses. On MNIST, epoch-wise efficiency drops more steeply and early-epoch maxima exceed $x(t)=At^2+Bt+C.$1; on CIFAR-10, decay is slower and the maximum is below $x(t)=At^2+Bt+C.$2 (Cueto-Mendoza et al., 2024). The paper attributes this to diminishing returns once performance plateaus. It further reports non-linear interactions among stopping criteria, model size, and efficiency. For example, on MNIST, LeNet’s mean efficiency spans from $x(t)=At^2+Bt+C.$3 under 50 epochs to $x(t)=At^2+Bt+C.$4 under the accuracy-bound criterion, whereas BCNN spans from $x(t)=At^2+Bt+C.$5 under early stopping to $x(t)=At^2+Bt+C.$6 under the energy-bound criterion (Cueto-Mendoza et al., 2024).

5. Inference-time relative efficiency and hardware fit

The 2023 efficient-computing work moves from training to inference and introduces a measurement-based composite score that trades off accuracy against measured energy per inference (Waltsburger et al., 2023). The primary definition is

$x(t)=At^2+Bt+C.$7

with the paper clarifying that “power consumption per inference (in mWh)” is energy per inference. In expanded form,

$x(t)=At^2+Bt+C.$8

and in the paper’s “physical-sense simplification” this becomes

$x(t)=At^2+Bt+C.$9

Operationally, accuracy is measured on the ILSVRC2012 validation set, while energy is computed from telemetry integrated over the inference window:

$x(t)$0

followed by

$x(t)$1

The monitoring stack combines NVIDIA DCGM for GPU power, RAPL via Scaphandre for CPU energy on x86, Node Exporter for system metrics, and Prometheus/Grafana for storage and visualization (Waltsburger et al., 2023).

The paper reports three full validation runs per model and hardware, plus stability tests with ten chained runs. On the A100 platform, MobileNetV3 Small achieves accuracy $x(t)$2, $x(t)$3, and score $x(t)$4, while EfficientNetV2-L achieves $x(t)$5, $x(t)$6, and score $x(t)$7 (Waltsburger et al., 2023). The score therefore makes explicit a relative efficiency judgment: a lower-accuracy model can be far more efficient than a higher-accuracy model when energy per inference differs by orders of magnitude. The same paper reports that MobileNetV3 Small remains best across all tested platforms and that some architectures, such as EfficientNetV2-L, are more strongly penalized off the A100, with scores dropping to $x(t)$8 on Quadro, $x(t)$9 on Bi-Xeon, and $B$0 on i7 relative to $B$1 on A100 (Waltsburger et al., 2023).

The authors describe this as a “fit test for neural architectures on hardware architectures.” The comparative logic is straightforward: for a fixed model and dataset, accuracy is unchanged across hardware, so cross-hardware score differences reduce to measured energy efficiency. A plausible implication is that inference-time relative neural efficiency can serve as a hardware-selection criterion just as directly as a model-selection criterion (Waltsburger et al., 2023).

6. Methodological issues, misconceptions, and limitations

A recurrent misconception is that relative neural efficiency denotes a single metric applicable across neuroscience and machine learning. The cited literature does not support that interpretation. In the neuroimaging paper, the formal quantities are $B$2, $B$3, and $B$4, and “relative neural efficiency” is only a grounded extension of that framework rather than a defined term in the paper (Steffener et al., 2016). In the 2024 training paper, by contrast, relative efficiency is explicitly a ratio of architecture-level efficiency scores (Cueto-Mendoza et al., 2024). In the 2023 inference paper, the relative comparison is score-based and may be normalized to a baseline hardware system for presentation, but that normalization is not part of the score definition (Waltsburger et al., 2023).

Several technical caveats are also domain-specific. In neuroimaging, the quadratic approximation is fitted across only four discrete load levels, and the differential-equation reinterpretation assumes smooth changes in slope with constant second derivative $B$5 (Steffener et al., 2016). The paper notes that sigmoidal models such as Weibull or logistic may capture richer inflections but require nonlinear fitting and more load points, and that the fixed block order may introduce sequence effects (Steffener et al., 2016). Moreover, $B$6 is defined operationally for capacity-limited cases with $B$7.

In the training-efficiency framework, the cost term is a sum of sampled power values rather than a physically explicit Joule integral, and the paper itself notes that a more physically consistent energy measure could refine units and thresholds (Cueto-Mendoza et al., 2024). The framework is also sensitive to stopping criteria and can be confounded by overtraining. The 100-epoch analysis shows that fixed long runs can distort relative comparisons: on CIFAR-10, the efficiency ratio between LeNet and BCNN falls to $B$8 under 100-epoch scoring, which the paper identifies as misleading because LeNet is overtrained while BCNN is comparatively less overtrained (Cueto-Mendoza et al., 2024).

In the inference-efficiency scoring literature, the score is explicitly inference-only and does not cover training efficiency (Waltsburger et al., 2023). The monitoring stack is hardware-dependent: CPU energy relies on RAPL for x86, and the GPU measurements shown are for NVIDIA via DCGM. The paper also notes that TensorFlow pre-allocates GPU memory, inflating occupancy statistics without necessarily altering energy totals; interpretation should therefore focus on measured energy rather than raw resource occupancy (Waltsburger et al., 2023).

Taken together, these limitations indicate that relative neural efficiency is best understood as a comparative methodology rather than a single invariant observable. In one setting it compares slopes and peaks of neural activation; in another it compares architectures by accuracy per logged training cost; in another it compares model–hardware pairs by accuracy per measured inference energy. The unifying principle is comparative conversion of demand or resources into a target outcome, but the observables, units, and inferential scope remain domain-specific (Steffener et al., 2016, Cueto-Mendoza et al., 2024, Waltsburger et al., 2023).