Performance-per-Dollar (PPD) Metric Analysis

• Performance-Per-Dollar (PPD) is a normalized metric that divides performance by cost to enable cost-effective comparisons across computing systems, ML models, and AI infrastructure.
• PPD is applied in various contexts including inference benchmarks, hardware pricing, and end-user performance, with tailored formulations for each use case.
• Empirical trends show rapid improvements in PPD and highlight trade-offs between cost efficiency and absolute performance, guiding strategic investments and optimization.

Performance-Per-Dollar (PPD) is a normalized metric that quantifies the efficiency of a system, model, or infrastructural element by dividing a target performance indicator by its associated dollar cost. The metric is applied across computing hardware, ML models, AI infrastructure, end-user devices, and benchmarking workflows to expose real-world cost-efficiency trade-offs and inform economically meaningful comparisons.

1. Core Definitions and Formalizations

PPD generically takes the form: $$\mathrm{PPD} = \frac{\text{Performance}}{\text{Dollar Cost}}$$ where "Performance" may denote throughput (e.g., FLOPs/s, tokens/s), accuracy (e.g., benchmark score), or an aggregated function (e.g., "Goodness" as defined in psychometric evaluation). "Dollar Cost" encompasses all dollar-denominated operational or capital expenses tied to the realization of that performance.

Specializations include:

• Machine learning inference: For inference or benchmark tasks,

$$\mathrm{PPD}_i = \frac{\text{Performance}_i}{\text{Cost}_i}$$

with performance as a normalized benchmark score and cost as total inference dollars required (Gundlach et al., 28 Nov 2025).

• Resource-normalized deep learning: For model evaluation,

$\mathrm{PPD} = \frac{P}{C_{\mathrm{model}}}$

where $P$ is predictive performance (e.g., accuracy) and $C_{\mathrm{model}}$ is the total incurred cost (energy plus GPU rental) (Selvan et al., 19 Mar 2024).

• Infrastructure and hardware: For GPU or system resource evaluation,

$$\mathrm{PPD}_{\mathrm{comp}} = \frac{\text{GFLOPs/s}}{\text{Dollars/s}}, \quad \mathrm{PPD}_{\text{bw}} = \frac{\text{GB/s}}{\text{Dollars/s}}$$

as used to diagnose compute versus bandwidth price trends (McDougall et al., 26 Sep 2025).

• End-user experience: Quantifies improvement/deterioration in user-perceived speed per unit cost, in units of “fractional speed change per \$” (Hastings et al., 2022):

$\mathrm{PPD} = \frac{P_1 / P_0 - 1}{C}$

• Composite metrics: For cross-layer AI infrastructure, PPD is a weighted ratio across multiple sub-metrics:

$$\mathrm{PPD} = \frac{\sum_{i} \alpha_i P_i}{\sum_j \beta_j C_j}$$

where $P_i$ and $C_j$ are selected performance and cost figures, and weights $\alpha_i, \beta_j$ tune the metric’s focus (He, 26 Nov 2025).

2. Methodological Variants and Benchmarking Protocols

Construction of PPD requires precise, context-specific accounting at both the performance and cost denominator:

• Inference and Large Model Benchmarks: For LLMs and AI benchmarks, benchmarking relies on standardized datasets (e.g., GPQA-Diamond), explicit reporting of tokens processed, normalized by input/output/cached token cost, and mapping to prevailing unit prices (via cloud, open or proprietary vendor) (Gundlach et al., 28 Nov 2025). Pareto-optimal selection along the performance-cost frontier is adopted to exclude dominated models.
• Deep Learning for Medical Imaging: The PePR framework generalizes PPD to multiple resource axes (energy, time, memory, emissions). Each resource axis can be monetized by physically measured units (Wh, GPU-hr) and then mapped to local cost basis (e.g., $/kWh,$/hr) (Selvan et al., 19 Mar 2024). Both normalized and absolute cost variants are reported.
• Feature-Based Hardware Pricing: For GPU compute markets, time-based billing is replaced by fine-grained resource metering—e.g., bandwidth-sampled cost measurement at sub-ms intervals, yielding a per-job charge directly tied to the relevant performance-limiting resource (McDougall et al., 26 Sep 2025).
• Infrastructure-wide Composite Metrics: Cross-layer analysis situates PPD in a unified taxonomy. The Metric Propagation Graph (MPG) models dependencies between energy, hardware, operations, and cost, enabling attribution of bottlenecks and optimization trade-offs (He, 26 Nov 2025).

3. Empirical Trends, Decomposition, and Key Findings

Empirical analyses across several domains yield multiple insights:

• Rapid PPD Gains in AI Inference: Gundlach et al. (Gundlach et al., 28 Nov 2025) report that PPD on frontier LLM benchmarks improved at rates of $5\times$–$10\times$ per year, with algorithmic efficiency alone contributing $3\times$ yearly improvement after controlling for hardware price decline. The observed annual cost declines and hardware-adjusted trends are benchmark-dependent, but consistently favor frontier and open-weight models.
• Resource-Equitable Learning Models: In medical imaging, fine-tuned, small-scale architectures dominate large models in both performance and PPD, with pretraining yielding substantial gains in both absolute accuracy and dollar efficiency. Resource-normalization exposes that raw performance gains may be overstated when operational cost is ignored (Selvan et al., 19 Mar 2024).
• GPU Hardware Economies/Disconnect: Feature-based pricing reveals that compute-per-dollar (FLOPs/\$) continues to scale, but bandwidth-per-dollar (GB/s/\$) stagnates or declines for newer GPU generations, underscoring that PPD must be resource- and workload-specific (McDougall et al., 26 Sep 2025).
• End-User Valuation of Speed: Human subject studies demonstrate small but non-negligible willingness-to-accept (WTA) compensation for performance losses, and yield empirical PPD figures ranging from 0.04 (fractional speed per dollar per day, incentive-compatible) to below 0.001 (lifetime loss, lab study), fundamental for user-centered design (Hastings et al., 2022).
• Cross-layer Propagation: Environmental shocks (e.g., grid carbon intensity rise) propagate systematically through the MPG, depressing downstream PPD—illustrated quantitatively by cluster-wide increases in cost and reductions in throughput (He, 26 Nov 2025).

4. Practical Considerations and Reporting Standards

Best practice for PPD computation and reporting includes:

• Explicit unit cost disclosure: Server, energy, bandwidth, token, or GPU-hour prices must be specified to enable apples-to-apples PPD comparison (Gundlach et al., 28 Nov 2025, Selvan et al., 19 Mar 2024, McDougall et al., 26 Sep 2025).
• Full denominator transparency: All cost components—including hidden or amortized operations costs (e.g., cooling, networking, staff, and carbon pricing)—should be made explicit as per infrastructure-wide frameworks (He, 26 Nov 2025).
• Performance normalization: When performance is multivariate (e.g., accuracy, throughput, latency), either report disaggregated PPDs or use multi-objective weighted aggregation with documented alpha/beta weights (He, 26 Nov 2025).
• Contextualization within Pareto frontier: Always identify and compare solutions on the non-dominated cost-performance front; dominated solutions artificially suppress aggregate PPD improvement rates (Gundlach et al., 28 Nov 2025).
• Benchmarking protocol documentation: Number of runs, token counts, provider and hardware descriptions, and pricing tier must fully accompany published PPDs to support auditability (Gundlach et al., 28 Nov 2025).

5. Limitations and Caveats

Several limitations constrain the interpretation of PPD:

• Non-monetary costs omission: Most PPD implementations exclude fixed costs (hardware acquisition, data curation), externalities (carbon offsets, water use), and temporal cost fluctuation (cloud spot rates, utilization-dependent pricing) (Selvan et al., 19 Mar 2024, He, 26 Nov 2025).
• Model selection bias: PPD maximization can favor models with moderate absolute performance but extreme cheapness, a potential issue if baseline accuracy is non-negotiable (Spangher et al., 28 Oct 2024, Gundlach et al., 28 Nov 2025).
• Methodological artifact: Human studies are sensitive to incentive compatibility, experience realism, and survey design, which may distort observed user value-of-performance (Hastings et al., 2022).
• Cross-domain and temporal inhomogeneity: PPD can vary by application domain, performance percentile, and time frame; infrastructure-induced bottlenecks can shift optimization targets (Gundlach et al., 28 Nov 2025, He, 26 Nov 2025).

6. Extensions and Multi-Objective Optimization

Sophisticated frameworks extend the PPD concept:

• Three-domain, six-layer taxonomy: Systematic propagation analysis ties changes in sustainability, cooling, hardware, runtime, and service operations into an integrated cost-performance optimization problem (He, 26 Nov 2025).
• Multi-objective frontiers: PPD serves as one axis in trade-off analysis alongside carbon-per-dollar and system reliability (MTBF/MTTR), enforced via Pareto-efficient exploration and Lagrangian-constrained optimization (He, 26 Nov 2025).
• Carbon- or energy-normalized PPD: By explicitly incorporating environmental externalities (e.g., grid carbon intensity) into the denominator, practitioners can define carbon-aware variants:

$$\mathrm{PPD}_{\mathrm{carbon}} = \frac{\sum_i\alpha_i P_i}{\sum_j \beta_j C_j + \lambda \sum_{i \in D_1} P_i \cdot \text{CI}}$$

enabling structural price-of-sustainability analysis.

• Lifecycle and capacity planning: Dynamic PPD computation supports mid-life upgrade evaluations, capacity allocation, and scenario planning under evolving price, power, and emissions constraints (He, 26 Nov 2025).

7. Implications and Strategic Impact

Systematic use of PPD disciplines progress claims in ML, AI, and computing infrastructure, guiding research investment, cloud procurement, hardware design, and public benchmarking. It directly answers the economic question “what capabilities are actually affordable, reproducible, and scalable?” Frontier research recommends that PPD be standardized alongside traditional metrics (accuracy, throughput, TCO, PUE) as a core Key Performance Indicator (KPI), ensuring that system engineering and model selection are constrained by cost-realism and broader resource equity (Gundlach et al., 28 Nov 2025, Selvan et al., 19 Mar 2024, He, 26 Nov 2025).