Data-Driven Encoding Scheme

• Data-driven encoding schemes are adaptive methods that derive representations from empirical data, task constraints, and deployment conditions.
• They expand the effective representation space by leveraging structured selection, continuous latentization, and geometry control for optimized performance.
• These schemes employ techniques like predictive modeling, numerical optimization, and adaptive selection to yield improvements in error rates, latency, and storage efficiency.

A data-driven encoding scheme is an encoding method in which the representation is determined, selected, or adapted using properties of the data, the task, the channel, the workload, or the deployment environment rather than by a fixed universal codebook. Across the literature, the term covers several distinct mechanisms: channel-aware message selection in deep-learning communication systems (Chen et al., 2018), confusion-aware multi-hot target assignment for neural classification (Jaiswal et al., 2019), learned selection of physical storage encodings in column stores (Cen et al., 2021), embedding-based binary representations for privacy-preserving record linkage (Vaiwsri et al., 1 Nov 2025), adaptive DNA strand coding based on local bit-pattern statistics (Li et al., 2021), direct numerical integration of Koopman inner products from samples (Ng et al., 2023), and hierarchy-preserving semantic identifiers for RDF inference (Curé et al., 2015). This breadth suggests that the unifying feature is not a single mathematical form, but the replacement of static, semantically blind, or distribution-agnostic encodings with representations whose geometry or selection depends on observed structure.

1. Conceptual scope and defining characteristics

In several lines of work, the immediate motivation for data-driven encoding is the inadequacy of fixed encodings. In deep-learning communication systems, the conventional $M\times 1$ one-hot vector can encode only $M$ messages, which imposes a limited data rate (Chen et al., 2018). In neural classification, one-hot encoding is fixed and task-agnostic, and any two distinct one-hot labels differ in exactly two bit positions, so the minimum Hamming distance is fixed at $H_{\min}=2$ (Jaiswal et al., 2019). In column stores, fixed defaults based only on data type and heuristics that mainly optimize storage footprint can fail because compression ratios and CPU decompression costs trade off differently on different hardware (Cen et al., 2021). In DNA storage, static encoding tables do not adapt well when real workloads contain changing local bit-pattern distributions (Li et al., 2021). In Koopman modeling, least-squares estimators such as EDMD are described as biased toward densely populated regions of state space (Ng et al., 2023).

These works therefore replace fixed encoding with mechanisms driven by empirical structure. In MUTE, codes are chosen according to how classes actually confuse one another in the dataset, with the objective

$$W_{\min} H_{\min} + \sum_{i=0}^{N-1}\sum_{j=i+1}^{N} W_{ij} H_{ij},$$

where $W_{ij}$ encodes class similarity and $W_{\min}=\frac{N(N-1)}{2}$ in the reported experiments (Jaiswal et al., 2019). In LEA, the advisor predicts the best encoding for each column slice or block by combining sample data and slice statistics such as cardinality or range, and can optimize for encoded size or query latency (Cen et al., 2021). In DP-DNA, the system counts the frequency of the four 2-bit patterns in each segment, identifies the lowest-frequency pattern, and selects the corresponding Digital Pattern-Aware Code for that segment (Li et al., 2021). In DDE for Koopman operators, the inner products defining the operator are estimated by weighted numerical integration over the dynamic range rather than by regression on sample density (Ng et al., 2023).

A useful distinction emerges from these examples. Some schemes are explicitly learned from data, as in LEA’s predictive models or MUTE’s optimization over class confusion (Cen et al., 2021, Jaiswal et al., 2019). Others are adaptive but not necessarily learned, such as the communication autoencoder’s channel-aware subset selection (Chen et al., 2018) or DP-DNA’s local pattern-aware strand selection (Li et al., 2021). Still others are analytic encodings designed to support downstream data-driven optimization, as in the photonic topological encoding method, which is described as data-driven because it provides a continuous latent space for machine-learning-based inverse design while not depending on training data to define that latent representation (Liu et al., 2019). This suggests that “data-driven” is used in the literature to denote at least learned, adaptive, and data-enabled analytic encodings.

Domain	Encoded object	Data-driven mechanism
DL communication	Message vectors	Adaptive vector selection; generalized data representation
Neural classification	Target labels	Multi-hot codes optimized from class confusion
Column stores	Column slices	Learned prediction of best physical encoding
DNA storage	Binary segments	Pattern-aware per-strand code selection
Koopman modeling	Observable inner products	Weighted integration from sampled state data
RDF inference	Ontology identifiers	Hierarchy-preserving semantic ID assignment

2. Representation-space expansion and geometry control

A recurring strategy is to enlarge the effective representation space without increasing the nominal dimensionality. In the generalized data representation scheme for deep-learning communication, the conventional one-hot vector is replaced by an $m$-order bit/probability vector with $m$ non-zero entries,

$\mathbf{s}={\underbrace {\left[\frac{1}{m} \quad 0\quad \cdots\quad 0\quad \frac{1}{m} \quad \cdots\quad \frac{1}{m} \quad 0\right]}_{\textrm{%%%%0%%%% non-zero entries}^T,$

so that the number of possible vectors becomes $\binom{M}{m}$ rather than $M$0 (Chen et al., 2018). The usable message count is taken as $M$1, and the rate becomes

$M$2

The paper reports that with $M$3 and $M$4, the rate is $M$5 bits/channel use versus $M$6 bits/channel use for one-hot, i.e. twice as high, while maintaining comparable BLER (Chen et al., 2018).

MUTE applies a related but label-space-oriented principle. It keeps the code length equal to the number of classes $M$7, uses $M$8 active bits per code, replaces the usual softmax with sigmoid activations, and trains each bit with binary cross entropy (Jaiswal et al., 2019). The optimization seeks both a large global minimum Hamming distance and larger Hamming distances for confusable class pairs. At test time, the sigmoid output is thresholded by keeping the top-$M$9 activated bits, and the predicted class is determined by Euclidean distance to the nearest code (Jaiswal et al., 2019). The intended effect is error correction in label space: a few corrupted bits need not move the output closer to a different class code.

Other schemes likewise exploit structured expansion or continuous latentization. KD encoding replaces one-hot symbol encoding with a $H_{\min}=2$0-way, $H_{\min}=2$1-dimensional discrete code $H_{\min}=2$2, where $H_{\min}=2$3, and composes the final embedding from position-specific code embedding matrices rather than a full $H_{\min}=2$4 lookup table (Chen et al., 2017, Chen et al., 2018). The parameter dependence on vocabulary size changes from linear to logarithmic, and experiments report reductions of 82%–97% in one paper and up to 98% in the other while maintaining similar or better downstream performance (Chen et al., 2017, Chen et al., 2018). In photonic inverse design, binary pixel topologies are mapped to a continuous sparse Fourier latent representation by first transforming the binary image into $H_{\min}=2$5, then applying the Fourier transform, and later reconstructing a binary structure by inverse transform and thresholding (Liu et al., 2019). The purpose there is dimensionality reduction and smooth optimization in latent space rather than direct compression of a discrete label set (Liu et al., 2019).

These examples share a geometric thesis: encoding quality is not exhausted by compactness. It also depends on whether the induced code space exposes discriminative margins, combinatorial diversity, smooth interpolation paths, or efficient parameter sharing.

3. Adaptive selection under channel, workload, and deployment conditions

Another major class of data-driven encoding schemes does not redesign the code alphabet itself, but selects among alternatives using current operating conditions. In the deep-learning communication system of (Chen et al., 2018), the adaptive transmission scheme first trains the autoencoder offline using all $H_{\min}=2$6 one-hot vectors at a fixed training SNR $H_{\min}=2$7, then probes each candidate vector online at the practical SNR $H_{\min}=2$8. The receiver evaluates the MSE of each vector after transmission, retains only those vectors whose MSE satisfies a threshold, forms a reduced set $H_{\min}=2$9 with $$W_{\min} H_{\min} + \sum_{i=0}^{N-1}\sum_{j=i+1}^{N} W_{ij} H_{ij},$$0, and maximizes

$$W_{\min} H_{\min} + \sum_{i=0}^{N-1}\sum_{j=i+1}^{N} W_{ij} H_{ij},$$1

At higher SNR, more vectors satisfy the MSE threshold; at lower SNR, fewer qualify, reducing rate but preserving robustness (Chen et al., 2018).

LEA provides a storage-systems analogue. It is trained in a two-part process: offline pre-training on synthetic data for data-dependent encoding behavior and in situ training on the target machine for hardware-dependent scan speed behavior (Cen et al., 2021). At inference time, LEA collects slice statistics such as range, cardinality, moments of adjacent differences for integral data, or cardinality and mean string length for string slices, computes the encoded size of a 1% contiguous sample for each supported encoding, predicts size and scan-speed properties, and chooses the encoding that performs best according to the selected objective (Cen et al., 2021). Compared with the heuristic-based advisor of a commercial column store on TPC-H, LEA achieves 19% lower query latency while using 26% less space, and stays within 10% of optimal across the tested configurations (Cen et al., 2021).

DP-DNA makes the adaptation even more local. Each strand includes an Encoding field in addition to primer, index, ECC, and payload, and the field is encoded using 11-code so that two nucleotides are enough to distinguish five modes: 00-code, 01-code, 10-code, 11-code, and 2bit-code (Li et al., 2021). For each segment, the system analyzes the 2-bit pattern distribution, evaluates whether 2bit-code is feasible without violating constraints such as fewer than 4 consecutive identical nucleotides and GC content between 40% and 60%, and may invoke a variable-length strategy to cut the segment early if that yields higher overall density (Li et al., 2021). The paper reports up to 103.5% higher encoding densities than prior work (Li et al., 2021).

This family of methods emphasizes that encoding may be part of an online control loop rather than a fixed preprocessing decision. A plausible implication is that, in such settings, the encoding scheme is better understood as a policy over candidate representations than as a single representation format.

4. Data-derived similarity, embedding, and semantics

Several data-driven encoding schemes derive their structure from empirical similarity or semantic relations. MUTE is explicit about this: classes that are frequently confused should be placed far apart in Hamming space, so that the network is forced to learn features that separate them more strongly (Jaiswal et al., 2019). The confusion matrix can be obtained either by inferencing a validation set using an already trained target model or by a class-wise autoencoder method cited in that work, then transformed into weights $$W_{\min} H_{\min} + \sum_{i=0}^{N-1}\sum_{j=i+1}^{N} W_{ij} H_{ij},$$2 by subtracting diagonal elements, finding the minimum error across triangles, thresholding large errors, and scaling the remaining non-zero values (Jaiswal et al., 2019). The paper reports consistent improvements over one-hot encoding and generally over Hadamard target encodings on MNIST, CIFAR-10, and ICON-50 across LeNet, ConvNet, AlexNet, DenseNet, ResNet, and ResNeXt, with particularly large gains for weaker architectures such as AlexNet (Jaiswal et al., 2019).

EmbBin for privacy-preserving record linkage also derives encodings from local textual structure, but at the q-gram level. It first generates the set of all possible q-grams $$W_{\min} H_{\min} + \sum_{i=0}^{N-1}\sum_{j=i+1}^{N} W_{ij} H_{ij},$$3, where $$W_{\min} H_{\min} + \sum_{i=0}^{N-1}\sum_{j=i+1}^{N} W_{ij} H_{ij},$$4, learns continuous embeddings for q-grams using Word2Vec’s CBOW model, then applies a learned/random projection and sign-based binarization to create temporary q-gram binary strings (Vaiwsri et al., 1 Nov 2025). Record-level binary strings are formed by aggregating the q-gram binaries with a bitwise-OR style rule, and the linkage unit compares candidates using the Dice coefficient

$$W_{\min} H_{\min} + \sum_{i=0}^{N-1}\sum_{j=i+1}^{N} W_{ij} H_{ij},$$5

The paper states that EmbBin generally performs well on short values, often outperforming TabHash and 2SH on short record values such as first names and first-plus-last names, while Bloom filters often perform best overall on linkage quality; EmbBin is also reported as more resistant to re-identification than Bloom filters and TabHash across all datasets, though less resistant than 2SH on long record values (Vaiwsri et al., 1 Nov 2025).

LiteMat uses semantic hierarchy itself as the signal that drives encoding. TBox concepts and properties are assigned structured identifiers so that if $$W_{\min} H_{\min} + \sum_{i=0}^{N-1}\sum_{j=i+1}^{N} W_{ij} H_{ij},$$6, then $$W_{\min} H_{\min} + \sum_{i=0}^{N-1}\sum_{j=i+1}^{N} W_{ij} H_{ij},$$7, enabling subclass and subproperty checks to be reduced to interval comparisons (Curé et al., 2015). The bound function computes the exclusive upper bound of the interval, and query evaluation becomes a numeric range test rather than full materialization or large UNION-based query rewriting (Curé et al., 2015). The evaluation reports ontology encoding times of 0.7 s for LUBM, 3.7 s for DBPedia, and 122 s for Wikidata, and ABox encoding throughput up to 2.8× faster than the compared standard encoding baseline (Curé et al., 2015).

Across these examples, the encoded geometry is not arbitrary. It is anchored in confusion, q-gram co-occurrence, or ontology subsumption. This suggests that a central property of many data-driven encoding schemes is semantic alignment between the induced code structure and the empirical or logical structure of the source domain.

5. Learning, optimization, and inference procedures

The optimization machinery used by data-driven encoding schemes varies widely, but several characteristic patterns recur. MUTE formulates code assignment as an integer linear programming problem and introduces the Narrow-convergence Approach heuristic: solve a simplified ILP for a short time, capture intermediate encodings, iteratively explore alternative encodings for a randomly chosen class, and repeat until no further improvement (Jaiswal et al., 2019). For a 10-class, 10-bit/4-hot case, the heuristic reaches essentially the same solution quality as ILP while reducing runtime by more than 99.75%, whereas CPLEX ILP took 11+ hours or even 50+ hours for some variants (Jaiswal et al., 2019).

KD encoding treats code learning as relaxed discrete optimization. Each discrete code dimension is represented by a one-hot vector relaxed by a temperature-controlled softmax, and a straight-through estimator is used so that the forward pass employs hard codes while the backward pass uses the softened approximation (Chen et al., 2017, Chen et al., 2018). One version first learns codes to reconstruct pretrained embeddings, while the later version embeds the scheme in end-to-end task training and adds Online Distillation Guidance and Pre-trained Distillation Guidance, with PDG reported as the strongest variant (Chen et al., 2018). These papers present code learning as a differentiable alternative to direct combinatorial search.

LEA uses supervised predictive modeling rather than direct combinatorial optimization. For each encoding type, it trains three models: one to predict encoded size, one to predict in-memory scan speed, and one to predict from-storage scan speed (Cen et al., 2021). Random forest regression is used for encoded size and in-memory scan speed, while linear regression is used both for strings longer than those seen during training and for from-storage scan speed to model latency and throughput of the storage device (Cen et al., 2021). The modeling pipeline is hierarchical: size prediction uses slice statistics and sample encoded size; in-memory scan speed uses predicted size and slice statistics; from-storage scan speed uses predicted size and predicted in-memory scan speed (Cen et al., 2021).

DDE for Koopman operators relies on numerical quadrature over partitions of the sampled dynamic range rather than supervised regression. The dynamic range $$W_{\min} H_{\min} + \sum_{i=0}^{N-1}\sum_{j=i+1}^{N} W_{ij} H_{ij},$$8 is partitioned, the entries of $$W_{\min} H_{\min} + \sum_{i=0}^{N-1}\sum_{j=i+1}^{N} W_{ij} H_{ij},$$9 and $W_{ij}$0 are approximated as weighted sums over node values,

$W_{ij}$1

and the Koopman matrix is then computed as $W_{ij}$2 (Ng et al., 2023). The paper proves convergence of these estimates under partition refinement and reports substantially lower total SSE and lower error variance than EDMD on trajectory data (Ng et al., 2023).

A common misconception is that data-driven encoding must be synonymous with deep learning. The surveyed work does not support that view. Integer programming, heuristic search, random forests, linear regression, numerical integration, and analytic hierarchy encoding all appear as central mechanisms (Jaiswal et al., 2019, Cen et al., 2021, Ng et al., 2023, Curé et al., 2015).

6. Performance claims, limitations, and recurring trade-offs

The empirical literature reports strong but domain-specific benefits. In communication, the adaptive transmission scheme reduces BLER by 80% relative to the conventional one-hot vector scheme when both operate at the same data rate, and the joint adaptive-plus-GDR scheme yields the best BLER across SNR regions in the reported simulations (Chen et al., 2018). In classification, MUTE improves average test accuracy over one-hot by 2.8% with LeNet and 7.1% with ConvNet on MNIST, by 41.5% with AlexNet and 3.6% with ResNet on CIFAR-10, and by 42.29% with AlexNet and 8.12% with ResNeXt on ICON-50 (Jaiswal et al., 2019). In column stores, LEA-Q provides the strongest latency improvements on StackOverflow and TPC-H, with 19% lower query latency and 26% less space than the heuristic-based advisor on TPC-H (Cen et al., 2021). In stream learning, GRF preprocessing often improves Kappa, with discussion values around +0.33 for MNB, +0.23 for SGD, +0.25 for Perceptron, and +0.25 for PA, though processing time always increases because the representation becomes larger (Lobo et al., 2019).

At the same time, the papers repeatedly identify nontrivial constraints. The communication autoencoder is sensitive to training SNR; training at a higher SNR improves convergence during training but can worsen BLER in practical use, while lower-SNR or multi-SNR training often yields better deployment robustness (Chen et al., 2018). MUTE requires offline code generation and solves a discrete optimization problem whose exact ILP form is computationally expensive (Jaiswal et al., 2019). EmbBin degrades on longer strings, partly because compressing from $W_{ij}$3 to $W_{ij}$4 can introduce noise and false positives (Vaiwsri et al., 1 Nov 2025). LEA assumes a uniform workload during training and inference, and incorporating concrete access patterns is described as future work (Cen et al., 2021). DDE notes that Delaunay triangulation becomes difficult for systems of order 8 or higher (Ng et al., 2023). DP-DNA incurs overhead for pattern scanning, feasibility checking, and the additional Encoding field, although the field is kept compact (Li et al., 2021).

A second misconception is that data-driven encoding always means improved compression alone. The surveyed work shows broader objectives: BLER reduction under rate constraints (Chen et al., 2018), robustness to noise and adversarial perturbations (Jaiswal et al., 2019), reduced cold-cache query latency (Cen et al., 2021), approximate matching with improved privacy (Vaiwsri et al., 1 Nov 2025), lower-dimensional continuous latent spaces for inverse design (Liu et al., 2019), or semantically complete RDF answering with minimal materialization (Curé et al., 2015). This suggests that the principal design question is not merely how many bits the code uses, but which task-relevant invariants, separations, or operational costs the encoding exposes.

The overall literature therefore portrays data-driven encoding schemes as a heterogeneous but coherent family of methods in which the representation is shaped by empirical structure, deployment conditions, or domain semantics. Their common departure from fixed encodings is clear; their differences lie in what counts as “data,” how adaptation is performed, and which system-level objective the encoding is designed to optimize.