Unified Experimental Design Framework

• Unified experimental design frameworks are comprehensive methodologies that integrate algorithmic, statistical, and computational approaches to optimize and analyze experiments across domains.
• They leverage innovations such as stochastic gradient methods, variance bounding, and convex relaxations to address diverse objectives including Bayesian, D/A/E-optimal, and combinatorial designs.
• These frameworks enable reproducible design specification using domain-specific languages and unify experimental and observational data for enhanced decision making.

A unified experimental design framework constitutes a rigorous, general-purpose methodology for optimizing, analyzing, or specifying experimental designs across diverse scientific and engineering domains. These frameworks provide algorithmic, mathematical, or programmatic infrastructure that subsumes a wide class of design objectives, estimator forms, randomization procedures, or experimental configurations. Recent advances have delivered unified frameworks for Bayesian-optimal experimental design, linear and generalized estimator variance bounding, p-norm and ratio objectives, programmatic design specification for human experiments, and minimax-adaptive strategies for combining observational and experimental evidence. This article surveys representative frameworks, with a focus on their technical principles, algorithmic innovations, and domain-agnostic applicability.

1. Unified Bayesian-Optimal Experimental Design: Stochastic Gradient Variational Framework

Recent work has unified Bayesian optimal experimental design (BOED) under a single stochastic gradient-based framework that directly maximizes variational lower bounds on the expected information gain (EIG) with respect to both design and variational parameters in a single loop (Foster et al., 2019). The EIG objective for any design $d$ is

$$U(d) = \mathrm{EIG}(d) = \mathbb{E}_{y \sim p(y|d)}\left[ D_\mathrm{KL}(p(\theta | y, d) \| p(\theta)) \right].$$

Rather than estimating EIG per candidate design and performing black-box optimization, the framework introduces variational lower bounds—most notably, the Barber–Agakov (BA) and the novel adaptive contrastive estimation (ACE) bounds—which are differentiable in both the variational guide $q_\phi(\theta|y, d)$ and $d$. Unified unbiased stochastic gradient estimators (via score function and reparameterization identities) enable end-to-end optimization via SGD variants, e.g., Adam, over both $\phi$ and $d$:

• BA bound:

$$L_\mathrm{BA}(\phi, d) = \mathbb{E}_{p(\theta)p(y|\theta,d)} [\log q_\phi(\theta|y,d)] + H[p(\theta)]$$

• ACE bound:

$$L_\mathrm{ACE}(\phi, d) = \mathbb{E}\left[\log \frac{p(y|\theta_0,d)}{\mathcal{Z}}\right], \quad \mathcal{Z} = \frac{1}{L+1} \sum_{\ell=0}^L \frac{p(\theta_\ell)p(y|\theta_\ell,d)}{q_\phi(\theta_\ell|y,d)}$$

This unified approach enables scalable BOED for high-dimensional design spaces (hundreds of parameters), as stochastic gradient methods efficiently jointly optimize over variational and design parameters. This single-loop methodology substantially outperforms traditional two-stage or gradient-free optimization especially in high-dimension (Foster et al., 2019).

2. Unified Variance Bounding for Linear Estimators in Arbitrary Designs

A fully general design-based inference framework subsumes variance bounding and estimation for any linear estimator in any experimental design, including difference-in-means, OLS, WLS, and their cluster-robust variants (Middleton, 2021). Any linear estimator can be reduced to a Horvitz–Thompson–type form with an appropriate contrast vector:

$$\hat{\tau} = c^\top W R y, \ \Longrightarrow \ \tilde{\tau} = 1_k^\top T^{-1} R z_c,$$

where the exact variance is $z_c^\top D z_c$, with $D$ the “design matrix” (determined by assignment covariances). Since $D$ cannot typically be identified from observed data, the framework systematically replaces $D$ with a positive semidefinite bound $\overline{D} \succeq D$ (e.g., generalized Neyman or Aronow–Samii bounds), enabling universally valid variance bounds.

A proposed unbiased estimator of $z_c^\top \overline{D} z_c$ uses reweighted assignment indicators:

$$\widehat{\mathrm{Var}}(\hat{\tau}) = z_c^\top R\, \overline{D}_p\, R\, z_c,$$

where division by the joint inclusion probability matrix $p = \mathbb{E}[R]$ is performed elementwise. This estimator unifies and justifies robust (Eicker–Huber–White/“sandwich”) and cluster-robust standard errors as special cases. Matrix spectral analysis provides a common ground for comparing designs and tightness of bounds. Thus, every linear estimator in any finite design is subsumed under (i) linearization, (ii) variance bounding, (iii) unbiased variance estimation, and (iv) direct design comparison via matrix spectra (Middleton, 2021).

3. General $p$-Norm Objectives and Interpolated Design Criteria

A single randomized local search algorithm is shown to solve experimental design for the full continuum of $p$-norm objectives, which interpolate between classical D-, A-, and E-optimality (Lau et al., 2023):

$$\Phi_p(M) = \left(\frac{1}{d} \sum_{i=1}^d \lambda_i^{-p} \right)^{1/p}, \quad \text{for } M = \sum_{i\in S} a_ia_i^\top$$

with $p=0$ (D-optimal), $p=1$ (A-optimal), $p\to\infty$ (E-optimal). The framework solves a convex relaxation, then performs randomized swaps (with two tuning parameters, $\gamma$ and $\kappa$) to steadily improve the objective:

• The algorithm matches best-known approximation bounds for each $p$, recovers D/A/E-design results as special cases, and interpolates smoothly between them depending on $p$.
• The algorithmic structure provides both performance guarantees and computational feasibility for interpolated (D/A/E) and intermediate-norm design formulations (Lau et al., 2023).

4. Combinatorial, Deterministic, and Local Search Unification

Unified local search—both combinatorial and rounding-based—offers end-to-end approximations for D-, A-, E-, and ratio-type objectives under knapsack or fairness constraints (Lau et al., 2020, Lau et al., 15 Oct 2024). Broadly:

• Deterministic rounding via the interlacing polynomial method delivers optimal or near-optimal approximation ratios for D/A/E/ratio objectives and small-$k$ (budget) settings (Lau et al., 15 Oct 2024).
• Randomized or deterministic local search, including Fedorov’s exchange and matrix-multiplicative-weights, supports all major design metrics, and extends to weighted, multi-knapsack, and fairness-constrained settings (Lau et al., 2020).
• All objectives are unified via a single-rounding or exchange algorithm, allowing provable error bounds, scalable runtime, and direct handling of multi-attribute or constrained experimental settings.

5. Unified Causal Discovery and Experimental Design for Cyclic and Acyclic Models

A two-stage, non-adaptive algorithm unifies experiment design for uniquely identifying both cyclic and acyclic causal graphs under the structural causal model framework (Mokhtarian et al., 2022). The method systematically combines:

• Stage 1: Recovery of descendants and strongly connected components (SCCs) via colored separating systems.
• Stage 2: Edge orientation through lifted separating systems, which are information-theoretically order-optimal (number of interventions $g+O(\log n)$ for $n$ variables, SCC size $g$).
• Generalization to bounded-intervention settings (fixed maximum intervention size) and extension to mixed graphs or latent confounding.

Unified separating systems and coloring strategies yield a single design methodology that applies to all simple SCMs, regardless of the presence or absence of cycles, and is minimally suboptimal with respect to the worst-case number or size of interventions (Mokhtarian et al., 2022).

6. Unified Programmatic and DSL-based Grammar for Design Specification

Recent advances in object-oriented and domain-specific languages provide a unified, compositional grammar for specifying experimental designs as programs (Tanaka, 2023, Bielicke et al., 14 May 2025). These frameworks encode unit specification, treatment crossing/nesting, randomization, trial-ordering, and assignment as operators in a formal grammar or DSL. Notably:

• The “grammar of experimental designs” (edibble package) defines a fixed pipeline of operations—set units, treatments, assignments, etc.—with a compositional data structure for the evolving design (Tanaka, 2023).
• PLanet’s DSL formalizes experimental design as a three-stage process: experimental-unit specification, trial-order construction (via algebraic operators: between/within/counterbalance/cross/nest), and unit-to-plan mapping. This structure enables mechanical verification, ambiguity reduction, and reproducibility of designs as code (Bielicke et al., 14 May 2025). Such DSLs abstract diverse experimental designs—from classic factorials to Latin squares to hierarchical blocks—under a small set of algebraic operators, supporting extensibility, error-checking, and reproducibility.

7. Decision-Aware, Multisource, and Label-Efficient Extensions

Unified frameworks have been extended to address downstream decision utility and complex data-source integration:

• An amortized Bayesian experimental design scheme parameterizes the experimental policy and decision function in a single neural process, unifying the experimental and decision phases for optimal downstream utility rather than information alone (Huang et al., 4 Nov 2024).
• Strategies for synthesizing experimental and observational data via minimax proportional regret unify sample allocation, site/arm selection, and shrinkage estimation, handling unknown bias and oracle reference via convex programming (Epanomeritakis et al., 27 Oct 2025).
• Label-efficient SFT of LLMs demonstrates that classical and new experimental design criteria (uncertainty, diversity, submodular facility location) are unified in a shared pipeline, delivering substantial annotation cost savings without the iterative overhead of classical active learning (Bhatt et al., 12 Jan 2024).

---

In summary, unified experimental design frameworks provide mathematically principled, algorithmically efficient infrastructure for (i) optimizing and analyzing experimental designs, (ii) bounding estimator uncertainty and bias, (iii) interpolating or extending classical criteria, (iv) programmatically specifying and reproducing design configurations, and (v) integrating diverse design objectives, including those tailored to modern machine learning, causal discovery, and policy evaluation (Foster et al., 2019, Middleton, 2021, Lau et al., 2023, Lau et al., 15 Oct 2024, Lau et al., 2020, Tanaka, 2023, Bielicke et al., 14 May 2025, Mokhtarian et al., 2022, Huang et al., 4 Nov 2024, Epanomeritakis et al., 27 Oct 2025, Bhatt et al., 12 Jan 2024). These frameworks are central to current research in statistics, causal inference, computational science, and machine learning, establishing the foundation for end-to-end experimental planning, analysis, and reproducibility across disciplinary boundaries.