Data Processing Efficiency (DPE)

• Data Processing Efficiency is defined as the effectiveness of converting resource expenditure into useful data transformation, evaluated through metrics like delay, energy, and storage overhead.
• It encompasses multiple formulations, from blockchain-enabled Metaverse systems to in-memory processing and Lakehouse ETL, making it inherently workload-dependent.
• Architectural strategies focus on reducing data movement and in-path overhead, leveraging near-data and in-memory computing to enhance overall system efficiency.

Data processing efficiency denotes the effectiveness with which a system converts resource expenditure into useful data transformation. Across recent literature, the term is instantiated in several technically distinct but structurally related ways: in blockchain-enabled Metaverse systems it is the ratio of processed data bits to weighted delay-and-energy cost; in analog in-memory inference it is the ability to keep more of a model’s computation inside the memory array while minimizing data movement and digital post-processing; in Lakehouse ETL it is evaluated through loading time and resulting table size; and in GPU and near-data systems it is tied to fusing or relocating execution so that less data traverses the storage–memory–host hierarchy (Qian et al., 2024, Zhao et al., 17 Nov 2025, Borodii et al., 23 Apr 2026, Ozawa et al., 11 May 2026, Vincon et al., 2019). Taken together, these works suggest that DPE is not a single universal scalar but a workload-dependent efficiency notion whose denominator may be dominated by delay, energy, storage overhead, host orchestration, or data movement.

1. Definitions and domain-specific scope

Recent work uses the term “data processing efficiency” to describe different optimization targets under a common systems premise: useful work should increase faster than resource cost. In RRAM-based in-memory computing, NL-DPE frames DPE as the ability to keep more of the model’s computation inside the memory array, thereby minimizing data movement and avoiding expensive digital post-processing (Zhao et al., 17 Nov 2025). In blockchain-enabled wireless Metaverse systems, DPE is explicitly introduced as “processed bits per unit of resource consumption,” with cost modeled through delay and energy (Qian et al., 2024). In Data Lakehouse benchmarking, DPE is treated operationally through ETL loading time and the file-system volume of the resulting tables, rather than through a single closed-form metric (Borodii et al., 23 Apr 2026). In heterogeneous acceleration and GPU query engines, DPE is bound to schedule quality, inter-device communication cost, kernel fusion, and the elimination of fragmented execution (Bai et al., 10 Feb 2025, Ozawa et al., 11 May 2026).

Domain	Operationalization	Representative source
Analog IMC for DNN inference	Keep more computation inside memory arrays; avoid ADCs and digital post-processing	(Zhao et al., 17 Nov 2025)
Blockchain-enabled Metaverse	Processed data bits divided by weighted delay-plus-energy cost	(Qian et al., 2024)
Data Lakehouse ETL	Data loading time and resulting table/storage size	(Borodii et al., 23 Apr 2026)
Heterogeneous CPU/GPU/FPGA execution	Data-aware scheduling under compute and communication constraints	(Bai et al., 10 Feb 2025)
GPU analytical query processing	Fused IO, decompression, and operators in a GPU-resident data path	(Ozawa et al., 11 May 2026)

A recurring misconception is to equate DPE with raw throughput alone. The surveyed literature does not support that simplification. Some formulations optimize throughput and energy jointly, some optimize storage footprint against ingest latency, and some emphasize how much intermediate data is avoided rather than how many arithmetic operations are issued. The common thread is not a specific unit, but a systems-level concern with reducing non-productive overhead.

2. Metric formulations and optimization objectives

The most explicit mathematical definition appears in the blockchain-enabled Metaverse formulation, where DPE is defined as

$$\text{DPE} := \frac{\text{processed data bits}}{\text{delay}+\text{energy}}.$$

For user-side local processing and server-side offloaded processing, the weighted costs are

$$cost^{(u)}_n = \omega_t T^{(up)}_n + \omega_e E^{(up)}_n,$$

and

$$cost^{(s)}_{n,m} = \omega_t \big(T^{(ut)}_{n,m}+T^{(sp)}_{n,m}+T^{(sg)}_{n,m}+T^{(bp)}_{n,m}+T^{(sv)}_{n,m}\big) + \omega_e \big(E^{(ut)}_{n,m}+E^{(sp)}_{n,m}+E^{(sg)}_{n,m}\big).$$

The corresponding optimization problem maximizes the sum of local and offloaded DPE ratios over user association, offloading ratio, bandwidth allocation, transmit power usage ratio, server computing allocation ratio, local computing usage ratio, and server-side data-processing versus blockchain-processing split (Qian et al., 2024).

That formulation is significant because it makes DPE a cross-layer objective rather than a purely communication-side or compute-side measure. The numerator is processed workload, but the denominator includes local computation delay, wireless uplink delay, server computation delay, block generation delay, block propagation delay, validation delay, and the associated energy terms. This explicitly separates DPE from spectral efficiency, energy efficiency, and throughput-only metrics, each of which omits at least one part of the end-to-end cost model (Qian et al., 2024).

Other papers adopt operational proxies rather than a single ratio. The Lakehouse benchmark evaluates DPE using two criteria only: data loading time and the volume of the tables formed in the file system. It does not introduce a separate composite DPE score or normalized formula; its logic is strictly comparative under identical Spark ETL conditions (Borodii et al., 23 Apr 2026). This matters because it shows that, in data engineering practice, DPE can be a multicriteria assessment rather than a scalar objective. A plausible implication is that DPE definitions remain workload- and system-specific even when the label is shared.

3. Architectural principle: minimizing data movement and in-path overhead

The broadest architectural thesis comes from near-data processing and processing-in-memory. “Moving processing to data” is presented as a response to the fact that modern data-intensive workloads are often dominated by data movement across the storage–memory–CPU hierarchy rather than by arithmetic itself. Near-Data Processing executes operations fully or partially in-situ close to the physical data location; Processing-in-Memory is a subclass of this model that targets computation directly within or very near memory chips. The stated goal is to transfer back only results rather than raw data, exploiting high internal device bandwidth and reducing latency, bandwidth consumption, and energy spent on movement (Vincon et al., 2019).

NL-DPE is a specialized realization of this principle for analog inference. Conventional RRAM crossbars compute the analog dot product

$\mathbf{y}=\mathbf{x}\mathbf{W},$

which is efficient for VMMs but inadequate for arbitrary non-linear functions and data-dependent matrix multiplications. NL-DPE augments crosspoint arrays with RRAM-based Analog Content Addressable Memory (ACAM), maps non-linear functions and DMMul-related subfunctions into decision trees, and places ACAM immediately after the crossbar so that analog outputs are consumed directly without standalone ADCs. The paper reports that ADCs can consume over 30% of area and more than 50% of power in conventional paths; in one comparison, the ACAM replacement incurs about 98% lower power and 86× smaller area than an 8-bit ADC plus activation logic. End-to-end, NL-DPE achieves 28× higher energy efficiency and 249× speedup over a GPU baseline, and 22× higher energy efficiency and 245× speedup over existing IMC accelerators, while maintaining high accuracy (Zhao et al., 17 Nov 2025).

The same movement-minimization logic appears in GPU database execution. Data Path Fusion fuses IO, decompression, and query operators into a single GPU kernel so that execution no longer fragments across CPU-orchestrated kernels with intermediate materialization in GPU global memory. In the reported prototype, the host is reduced to plan generation and kernel launch, while the GPU remains in control throughout the data path. The resulting system reports speedups of 2.66 to 6.22 on TPC-H and 3.84 to 16.81 on SSB over the state-of-the-art approach in the representative configuration (Ozawa et al., 11 May 2026).

SkimROOT applies a near-storage variant of the same idea to LHC skimming. Instead of transferring large ROOT baskets to remote compute nodes for decompression, deserialization, and filtering, SkimROOT runs filtering on an NVIDIA BlueField-3 DPU attached to the storage server and returns only the reduced output. In the reported Higgs-analysis evaluation at 1 Gbps, client-side LZ4 filtering takes 382.1 s, whereas SkimROOT takes 8.62 s, corresponding to a 44.3× speedup (Batsoyol et al., 4 Jun 2025). Across these systems, DPE improvements arise less from isolated kernel optimization than from reducing unnecessary traversal of the data path.

4. Workload-aware scheduling, precision adaptation, and resource allocation

A second major theme is that DPE depends on matching execution strategy to the structure of the data. DyPe formalizes this for irregular workloads on heterogeneous GPU–FPGA systems. Rather than assuming a manually fixed partition, it analyzes input characteristics such as sparsity, input dimensions, arithmetic intensity, sequence length, and window width, then explores a multi-objective, multi-constraint design space over mappings of kernels to devices and groupings of kernels into pipeline stages. Its dynamic-programming states take the form $dp[i][f][g]$, representing the best schedule for the first $i$ kernels using $f$ FPGAs and $g$ GPUs. On real hardware, conventional static scheduling is optimal in only 13 out of 86 cases, whereas DyPe finds the optimal schedule in 77 out of 86 cases, with an average of only 3.95% performance or energy-efficiency loss in the sub-optimal cases. It yields on average 1.53× throughput and 1.09× energy efficiency over the static schedule baseline, and 1.44× throughput and 1.66× energy efficiency over the GPU-only baseline (Bai et al., 10 Feb 2025).

At the arithmetic-datapath level, DHFP-PE improves DPE by matching the processing element to the precision actually required. Its 6-stage pipelined MAC engine supports FP8 (E4M3, E5M2) and FP4 (2 × E2M1, 2 × E1M2) by reusing a single 4-bit unit multiplier. In FP8 mode the multiplier operates as a standard $4\times4$ block; in FP4 mode the same physical array is partitioned into two simultaneous $2\times2$ multipliers. The reported 28 nm implementation operates at 1.94 GHz with 0.00396 mm$$cost^{(u)}_n = \omega_t T^{(up)}_n + \omega_e E^{(up)}_n,$$0 area and 2.13 mW power, with up to 60.4% area reduction and 86.6% power savings compared to state-of-the-art designs. Reported throughput and efficiency are 3.88 GFLOPS and 1818 GFLOPS/W in FP8 mode, and 7.75 GFLOPS and 3632 GFLOPS/W in FP4 mode (Kumar et al., 6 Apr 2026).

In wireless blockchain-enabled Metaverse systems, workload adaptation is expressed through joint resource allocation rather than device scheduling. The DAUR algorithm alternates optimization of user association, work offloading ratios, task-specific computing resource distribution, bandwidth allocation, user power usage ratios, and server computing resource allocation ratios. The paper reports that DAUR achieves the best DPE among the tested methods, with 86.48 M bits/(s·J) in the conference version and 87.87 M bits/(s·J) in the journal version, and that the algorithm converges quickly in practice (Qian et al., 2024, Qian et al., 7 Jul 2025). The shared systems insight is that DPE is not fixed by hardware capability alone; it is shaped by runtime matching between data properties, precision, topology, and allocation policy.

5. Data engineering and scientific-operations instantiations

In Data Lakehouse architectures, DPE is measured at the ETL and storage layer rather than at the arithmetic or kernel level. The benchmark compares Apache Hudi, Apache Iceberg, and Delta Lake under the same Apache Spark workflow: four sequential ETL processes, identical transformations, and identical input datasets spanning structured CSV and semi-structured JSON up to 7 GB. Delta Lake is the fastest overall for loading tasks. On the largest integrated table, weather_hourly_events, Delta loads in 285.18 seconds, compared with 388.55 seconds for Iceberg and 737.03 seconds for Hudi; for air_quality_level, the corresponding times are 9.57 s, 10.7 s, and 69.33 s (Borodii et al., 23 Apr 2026).

Storage efficiency tells a different story. Apache Iceberg consistently produces the smallest tables: for weather_hourly_events, 270 MB versus 306 MB for Delta Lake and 4.41 GB for Hudi; for weather_events, 248 MB versus 286 MB and 2.26 GB; for air_quality_level, 17 MB versus 19.7 MB and 474 MB (Borodii et al., 23 Apr 2026). The immediate implication is that DPE in analytical data systems can be multi-objective and format-dependent: Delta Lake is recommended when fast loading of data of any volume is the priority, while Apache Iceberg is recommended when disk space savings and stability are more important. The benchmark also states that Hudi is ineffective for the loading and storage tasks studied here, but may still be effective in incremental update and streaming processing scenarios that were not experimentally evaluated (Borodii et al., 23 Apr 2026).

Scientific monitoring systems show yet another DPE regime: sustained operational responsiveness under continuous ingest. The Daya Bay onsite Performance Quality Monitoring system runs on an 11-server offline environment, processes about 320 raw files per day of about 1 GB each from three experimental halls, and uses database polling, file-transfer status checks, PBS job submission, NuWa reconstruction, histogram generation, and web publication. The timing budget is explicit: file transfer takes less than 5 minutes, job submission about 10 seconds, NuWa processing about 30 minutes, and histogram printing plus web publishing about 2 minutes, producing a typical total latency of about 40 minutes after a raw data file is closed (Liu et al., 2014). Efficiency here derives from incremental per-file processing, dedicated and elastic compute resources, and dynamic histogram creation that reduces output ROOT file volume and avoids unnecessary pre-allocation.

6. Trade-offs, limitations, and terminological ambiguity

The surveyed literature consistently treats DPE as workload-dependent and often explicitly bounded by trade-offs. The Lakehouse benchmark is limited to loading and storage; it does not evaluate read performance, query latency, update/merge/delete operations, or streaming behavior, and its results are tied to a single-node local Spark environment (Borodii et al., 23 Apr 2026). DPF achieves substantial speedups, but the authors note that joins are supported only when the build-side hash table fits in GPU memory, IO is restricted to raw block devices through BaM, only GPU-FOR and FSST are integrated as representative compression schemes, the loader is still host-side, and multi-GPU support is left for future work (Ozawa et al., 11 May 2026). SkimROOT is presented as a prototype-stage system evaluated single-threadedly on one core, with future work aimed at better prefetching, more parallelization, and scaling across multiple DPUs (Batsoyol et al., 4 Jun 2025).

Algorithmic DPE optimization also comes with non-global guarantees. The DAUR framework converts a mixed-integer sum-of-ratios problem into alternating convex subproblems using fractional programming, SDR, DC refinement, and discrete recovery, but the reported convergence target is a stationary point rather than a global optimum (Qian et al., 2024). More generally, the near-data processing review emphasizes that effective DPE gains require new architectures, ISAs, virtual-memory support, coherence handling, programming models, and toolchains; without these, movement reduction may remain limited to specialized cases (Vincon et al., 2019).

The acronym itself is not semantically unique. In efficient code-generation evaluation, DPE denotes “Differential Performance Evaluation,” a framework that builds performance-exercising tasks and assigns a Differential Performance Score (DPS) relative to clustered reference solutions; its benchmark, EvalPerf, contains 121 tasks (Liu et al., 2024). In interactive segmentation, DPE denotes “Dynamic Prompt Embedding,” a mechanism that processes only a region of interest and can reduce latency to below 25% of full prompt embedding for predominant small objects while having marginal impact on accuracy (Huang et al., 13 Jul 2025). These usages are terminologically separate from data processing efficiency proper. Their coexistence underscores a practical point: in current arXiv literature, “DPE” is an overloaded acronym, whereas “data processing efficiency” remains a systems-level concept whose concrete meaning is specified by the workload, cost model, and execution substrate under study.