Training-Testing Co-Design

• Training-testing co-design is a methodology that jointly optimizes training objectives and evaluation conditions to ensure realistic benchmarking and efficient deployment.
• It employs design patterns like closed-loop workload synthesis, mutual distillation, and surrogate modeling to align model performance with system constraints.
• This approach has practical applications in many-core systems, model compression, microbial inference, and robotic sim-real learning to improve overall accuracy and efficiency.

Training-testing co-design denotes a class of methodologies in which training-time objectives, representations, budgets, and search procedures are explicitly coupled to the conditions under which a system will later be validated, benchmarked, or deployed. In the literature, this coupling appears in several distinct but structurally related forms: many-core co-design automation embeds a user-customizable workload generator into the design pipeline rather than treating benchmarking as a downstream afterthought (Dhanasekar et al., 2018); language-model distillation modifies the training loop so that the final deployed student is both smaller and more accurate than standard one-way knowledge distillation typically yields (Lee et al., 2023); microbial network inference uses a single cross-validation framework for both hyper-parameter selection and held-out comparison of algorithms (Agyapong et al., 2023); and sim-real robot learning uses reinforcement learning in simulation while retaining an auxiliary supervised loss on real data so that improvements in simulation remain aligned with real-world success (Shi et al., 13 Feb 2026). Closely related work extends the same logic to supernet-based accelerator search, training-aware transformer and accelerator co-design, training-aware modeling of heterogeneous dataflow accelerators, and uncertainty-aware optimization from training through fabrication (Fan et al., 2021, Tuli et al., 2023, Morlier et al., 16 Mar 2026, Du et al., 3 Jun 2026).

1. Conceptual scope

At its most general, training-testing co-design treats training and evaluation as a coupled optimization problem rather than two independent phases. The coupling may target benchmark realism, hyper-parameter selection, deployment efficiency, real-world transfer, or manufacturing-aware system feasibility. In the many-core setting, co-design automation integrates the application complexity design space, heterogeneous many-core architecture design space, parallel programming language design space, simulator design space, and optimizer design space, while also requiring a built-in workload generator that can benchmark both the whole system and individual subsystems such as caches, main memory, networks, schedulers, and functional units (Dhanasekar et al., 2018). In this formulation, the benchmark generator itself becomes part of the design pipeline.

A second interpretation appears in model compression and mutual learning. Co-Training and Co-Distillation replaces the conventional frozen-teacher pipeline with bidirectional co-training so that training-time interactions are deliberately engineered for the quality and efficiency of the final deployed student (Lee et al., 2023). A third interpretation appears in statistical model assessment: microbial co-occurrence network inference is reframed as a prediction problem on held-out samples, so the same predictive criterion drives both sparsity selection during training and generalization assessment during testing (Agyapong et al., 2023).

The same principle extends beyond supervised learning. In RL-based sim-real co-training for vision-language-action models, simulation is not used merely as a static source of demonstrations; it is used as a closed-loop interaction environment for reinforcement learning, while real-world data remains active as an anchoring term during policy updates (Shi et al., 13 Feb 2026). Training is therefore organized around the deployment distribution rather than around simulation performance alone.

A further extension is training-aware runtime approximation. FPSAttention does not stack FP8 quantization and sparsity as a training-free inference trick; it jointly learns low-precision, sparse attention behavior during training so that the model becomes robust to the exact approximations used later at inference (Liu et al., 5 Jun 2025). At the most expansive end of the spectrum, end-to-end processor co-design treats network architecture and training budget, chip mapping, fabrication yield, and compute-resource allocation as one compositional optimization problem, with uncertainty modeled as an explicit resource (Du et al., 3 Jun 2026).

2. Recurrent design patterns

One recurrent pattern is closed-loop workload synthesis. The user-customizable workload generator in many-core co-design can produce workloads with specified levels of computation complexity, communication complexity, control-flow complexity, and locality of reference under user-specified distributions, and it also supports computational surges and communication surges (Dhanasekar et al., 2018). This allows characterization, synthesis, benchmarking, and validation to share a common workload model rather than rely on static benchmark suites.

A second pattern is mutual optimization of train-time participants for a test-time artifact. CTCD is emblematic: the student helps the teacher through reversed knowledge distillation, and the improved teacher then helps the student. The paper reports that the small model distilled by CTCD outperforms the original larger model by 1.66 points on average on GLUE, while the teacher gains by 1.88 average points, and the teacher is not needed at test time, so inference cost is unchanged relative to a standard small distilled model (Lee et al., 2023).

A third pattern is decoupled expensive training and cheap evaluation via surrogates. In reconfigurable CNN accelerator co-design, a supernet is trained once using progressive shrinking, then Gaussian process models predict cross-entropy loss, latency, and energy so that genetic search can proceed without repeated network training or repeated synthesis and place-and-route (Fan et al., 2021). TransCODE applies the same logic to transformers and ASIC accelerators: DynaProp provides dynamic pruning during training and inference, ELECTOR simulates a design space of accelerators, and the co-design search jointly optimizes accuracy, latency, energy, and area (Tuli et al., 2023).

A fourth pattern is training-time alignment with deployment distribution. RL-Co warm-starts on a mixture of real and simulated demonstrations, then fine-tunes in simulation with reinforcement learning while maintaining an auxiliary supervised loss on real data (Shi et al., 13 Feb 2026). The training objective is thus explicitly shaped by the requirement that the policy remain effective on the real robot.

A fifth pattern is uncertainty as a first-class design variable. In uncertainty-aware neural-network-processor co-design, success probabilities from training, mapping, and fabrication are not collapsed into point estimates; instead, Confidence, defined as the inverse of success probability, is optimized alongside cost, time, and power (Du et al., 3 Jun 2026). This shifts reliability from a post-hoc diagnostic to a design knob.

3. Formal mechanisms

The formalization of training-testing co-design varies by domain, but several representative formulations recur. In workload-centric many-core co-design, the workload is modeled using a graph-theoretic or hypergraph-based representation in which nodes correspond to algorithms or operations and weighted edges represent communicated data scaled by dependency depth. One key equation is

$e_{pq} = d_{pq} D_{pq},$

where $e_{pq}$ is the hyper-edge weight between adjacent vertices $p$ and $q$, $D_{pq}$ is the data size transferred, and $d_{pq}$ is the depth index defined as the difference in levels of the communicating vertices. Control-flow complexity is modeled by a probability vector $\{p_1, p_2, \dots, p_n\}$ over fan-out paths, and locality of reference is varied through a randomized loop/index model so that spatio-temporal locality becomes controllable rather than incidental (Dhanasekar et al., 2018).

In mutual distillation, the coupling is encoded directly in the objective. CTCD defines teacher parameters $\theta$ and student parameters $\phi$, with the standard teacher-to-student term

$$\mathcal{L}_{KD:\, t\to s}(\theta,\phi) = \alpha_h \, H(q,p_{\phi}) + \alpha_s \, D(p_{\theta},p_{\phi}),$$

the reverse student-to-teacher term

$e_{pq}$0

and the combined objective

$e_{pq}$1

The defining feature is that teacher and student are jointly optimized through mutual distillation rather than through a fixed one-way transfer (Lee et al., 2023).

In predictive evaluation for network inference, the design is procedural rather than tied to a single compact loss. The proposed method is a $e_{pq}$2-fold cross-validation framework, illustrated with $e_{pq}$3: split into train and test folds, split the training fold again into subtrain and validation, fit on subtrain, tune hyper-parameters by validation error, refit on the full training set, evaluate held-out test error, and average across folds. The analysis is repeated $e_{pq}$4 times, each time using a different taxon as the outcome variable and the remaining taxa as predictors, and mean squared error on the test set is the primary prediction loss (Agyapong et al., 2023). The same predictive scoring rule is therefore used for both model selection and algorithm comparison.

In uncertainty-aware end-to-end co-design, the stochastic nature of training and downstream realization is formalized explicitly. The training block models accuracy as a stochastic process over architecture, setting, and epoch budget, and success probability for a target accuracy becomes a resource. The central abstraction is

$e_{pq}$5

where $e_{pq}$6 is success probability. Smaller values therefore correspond to higher reliability, and confidence composes multiplicatively across independent stochastic stages (Du et al., 3 Jun 2026). This formulation is noteworthy because it embeds uncertainty directly in the optimization algebra.

4. Systems and hardware realizations

A large fraction of the literature instantiates training-testing co-design as hardware-software or algorithm-system co-design. In reconfigurable CNN acceleration, the three-phase framework of Specify and Train, Modeling, and Exploration decouples DNN training from joint architecture-hardware exploration, uses a hardware-friendly neural architecture space based on ResNet-50-like residual blocks, and relies on Gaussian process models to predict loss, latency, and energy. The paper reports that finding one optimized design takes only 0.1 GPU hour and that, compared with manually designed ResNet101, InceptionV2, and MobileNetV2, the method can achieve up to 5% higher accuracy with up to 3x speed up on ImageNet (Fan et al., 2021).

Transformer co-design extends this pattern to a larger model-hardware space. TransCODE combines DynaProp, which prunes weights, activations, and gradients at runtime, with ELECTOR, a simulator spanning 14,850,000 accelerators, and a weighted performance objective over accuracy, latency, dynamic energy, leakage energy, and area. The reported best transformer-accelerator pair achieves 0.3% higher accuracy than the state-of-the-art pair while incurring 5.2$e_{pq}$7 lower latency and 3.0$e_{pq}$8 lower energy consumption (Tuli et al., 2023). MONET, by contrast, targets training explicitly rather than inference-derived approximations of training. It models forward pass, backward pass, optimizer update, layer fusion, and activation checkpointing on heterogeneous dataflow accelerators, and reports that for ResNet-18 training with Adam, batch size 1, and $e_{pq}$9 inputs, a checkpointing configuration can save up to 13 MB of memory at only about 4% extra latency and energy (Morlier et al., 16 Mar 2026).

Training-side structure shaping is also used to improve runtime behavior directly. BalanceGS modifies 3D Gaussian Splatting training so that Gaussian density becomes more balanced, then adjusts GPU scheduling and memory mapping to match the induced distribution. The abstract reports a 1.44$p$0 training speedup on a NVIDIA A100 GPU with negligible quality degradation (Wu et al., 16 Oct 2025). In N:M sparse training, BDWP applies bidirectional weight pruning while SAT provides a sparse training accelerator and supporting dataflow optimizations; under a 2:8 sparse ratio, the accelerator achieves an average speedup of 1.75x over dense training, accompanied by a negligible accuracy loss of 0.56% on average (Fang et al., 2023).

Recommendation training provides a distinct systems case because sparse embedding layers dominate. Tensor Casting identifies gradient expand-coalesce as a training bottleneck, recasts it as gather-reduce, and pairs that transformation with a memory-centric tensor gather-scatter architecture. When prototyped on a real CPU-GPU system, Tensor Casting provides 1.9-21x improvements in training throughput compared to state-of-the-art approaches (Kwon et al., 2020). A larger-scale distributed counterpart is the software-hardware co-design of Neo and ZionEX for DLRM training: the system trains models with up to 12 Trillion parameters and reports a 40X speedup in time to solution over previous systems (Mudigere et al., 2021). By contrast, the Manoa cluster, MiMatrix job server framework, and GDRAA collective are explicitly training-focused rather than joint training-inference co-design, but they still demonstrate the same principle that communication algorithms, scheduling, and cluster topology must be designed around the training workload; the system uses 128 P100 GPUs and GDRAA requires two synchronizations in each iteration, which the paper describes as a theoretical minimum (Chen et al., 2018).

5. Domain-specific instantiations

The topic is not restricted to accelerator research. In workload characterization for many-core systems, the User Customizable Workload Design Process assembles workloads from an ALGOBANK containing numeric, semi-numeric, non-numeric, and general-purpose algorithms. Because the workload model supports application cloning, computational surges, and communication surges, it can be used both for benchmarking synthesized architectures and for generating stand-ins when the original application cannot be exposed for intellectual-property or regulatory reasons (Dhanasekar et al., 2018). This suggests that, in benchmark-centric settings, the test workload itself is a designed object.

In microbial data analysis, the central issue is fair evaluation without ground truth. The cross-validation framework for co-occurrence network inference converts Pearson correlation, Spearman correlation, Gaussian Graphical Model, and LASSO into predictors on held-out data so that all methods can be scored on the same MSE scale (Agyapong et al., 2023). The paper reports that Yeo-Johnson transformation plus standard scaling improves prediction accuracy over scaling alone on Amgut2, that accuracy generally improves as sample size increases, and that LASSO and GGM are generally the most accurate methods among those studied, although the preferred method depends on dataset characteristics and the biological question (Agyapong et al., 2023).

In antibody structure-sequence co-design, the Hierarchical Training Paradigm is explicitly organized around the downstream conditional-generation task. Stage I uses ESM-2 on large-scale protein sequences; Stage II adapts to antibody sequences from OAS; Stage III performs pocket inpainting on protein-protein complexes from DIPS; and Stage IV performs antibody-antigen co-design on SAbDab (Wu et al., 2023). The staging proceeds from decreasing data abundance to increasing task specificity, aligning training progressively with the actual test-time condition in which the antigen structure and antibody framework are known while the CDR is missing. The full HTP achieves H3 AAR of 40.98, RMSD of 2.06, and TM-score of 0.9621 in the reported ablation table (Wu et al., 2023).

Robotic sim-real co-training is another clear instance. RL-Co is evaluated on four real-world tabletop manipulation tasks with OpenVLA and $p$1, using ManiSkill for simulation and a Franka Emika Panda robot for deployment. The paper reports real-world average success of 64.0% for OpenVLA under RL-Co versus 40.0% for SFT co-training, and 66.2% for $p$2 versus 45.9%, corresponding to the reported +24% and +20% improvements (Shi et al., 13 Feb 2026). The same study reports that, in some settings, only 20 real-world demonstrations under RL-Co can match or exceed the performance that real-only or SFT co-training obtains with 200 demonstrations (Shi et al., 13 Feb 2026). Here the training-testing coupling is measured directly in deployment success and data efficiency.

6. Misconceptions, limitations, and emerging directions

A common misconception is that training-testing co-design is simply inference acceleration with some training details added later. Several papers reject this directly. MONET argues that inference-only tools do not capture the larger and more irregular design space of training, where the backward pass, optimizer state, activation storage, recomputation, and training-specific operator patterns all matter (Morlier et al., 16 Mar 2026). The broader implication is that inference-oriented intuitions often do not transfer.

A second misconception is that benchmarking can remain static while system design evolves. The many-core workload paper states that existing benchmark suites such as SPEC, LINPACK, IBS, SPLASH-2, PARSEC, and Rodinia are tied to fixed applications or kernels, become obsolete over time, and do not provide the fine-grained customizability needed to tune computation, communication, control-flow, and locality independently, nor do they naturally generate computational or communication surges (Dhanasekar et al., 2018). In that setting, static benchmarks are themselves a source of train-test mismatch.

A third misconception is that any plausible validation proxy is sufficient. In microbial network inference, network consistency across sub-samples can reward degenerate models that infer no edges, since an empty network is perfectly consistent across splits (Agyapong et al., 2023). In language-model compression, one-way co-training is not enough; the CTCD experiments explicitly compare stand-alone training, co-training plus one-way distillation, and co-training plus co-distillation, and report that the bidirectional distillation in CTCD is what produces the best student (Lee et al., 2023). In sim-real robotics, the ablations show that removing real-data regularization during Stage II can drop Pick and Place success with $p$3 from 81.38% to 40.25%, while removing real supervision from Stage I drops final performance to 12.5%, and removing real supervision from both stages drops success to 6.25% (Shi et al., 13 Feb 2026). These results make explicit that evaluation-aligned training signals are not interchangeable.

A further conceptual shift is the treatment of uncertainty. In uncertainty-aware processor co-design, Confidence is presented as a continuously tunable design knob rather than a post-hoc diagnostic, and the case studies show that tightening functionality constraints shifts the Pareto boundary outward toward higher cost and computation time (Du et al., 3 Jun 2026). This suggests a broader interpretation of testing: not merely measuring nominal performance, but specifying the probability with which requirements must be met.

Emerging directions widen the scope from performance to system lifecycle. The edge-AI survey argues that training precision, optimizer design, adaptive inference, benchmarking, compiler behavior, and hardware support should be coordinated across the development stack, and it identifies hidden cross-layer opportunities such as combining on-device training methods, integrating adaptive inference with online adjustment, and extending benchmarking to flexible hardware (Hao et al., 2021). The sustainable-training survey argues that performance-per-watt, mixed precision, kernel fusion, checkpointing, energy-aware scheduling, and standardized energy reporting should be treated as co-design targets across the full training stack, and it estimates a combined software uplift of about 4$p$4 performance-per-watt from mixed precision, kernel fusion, checkpointing, and power capping (Makin et al., 28 Jul 2025). Taken together, these directions indicate that training-testing co-design is evolving from a deployment-efficiency heuristic into a general methodology for aligning optimization, evaluation, and resource accounting across the full model and system lifecycle.