All@Once: Unified Joint Formulation

• All@once is a strategy where variables, objectives, and constraints are optimized jointly, replacing sequential processing with a unified computation.
• It enables simultaneous treatment of large-scale problems, recovering dependencies via single-pass estimation, and supports techniques like train-once, deploy-many.
• The method enhances resource efficiency and covariance recovery in areas ranging from astronomical imaging and inverse problems to model compression and reinforcement learning.

“all@once” is a compact label for a recurring research strategy in which objects, variables, goals, constraints, or measurements that are usually processed sequentially are instead handled in a single joint formulation. In the cited literature, the expression appears in forms such as “fit everything everywhere all at once” for astronomical photometry, “Pruning Everything, Everywhere, All at Once” for structured model compression, “Once-for-All” for train-once deployment frameworks, “Every Benchmark All at Once” for quantum characterization, and all-at-once formulations for inverse problems and time-dependent PDE solvers (Stone et al., 2023, Nascimento et al., 4 Jun 2025, Ito et al., 2023, Silva et al., 21 Oct 2025, Kaltenbacher, 2016). This suggests that “all@once” is not a single algorithmic family but a broader methodological stance: collapse repeated subproblems into one differentiable, algebraic, or statistical object, then recover the desired per-object or per-budget outputs by indexing, post-processing, or constrained selection.

1. Semantic scope and recurrent meanings

Across the supplied literature, “all@once” has several technically distinct but structurally related meanings. One meaning is literal joint optimization over all relevant unknowns. AstroPhot defines “everything” as sky/background, stars, galaxies, and the point-spread function, “everywhere” as every native pixel in each exposure, and “all at once” as a single objective over all exposures and pixels (Stone et al., 2023). In inverse problems, the same phrase denotes treating model and observation equations as one coupled system rather than eliminating the state through a parameter-to-state map (Kaltenbacher, 2016).

A second meaning is train once, deploy many times. In Once-for-All neural architecture search, the expensive supernet training is done only once, after which multiple subnetworks can be extracted according to deployment scenarios; OFA$^2$ extends this by making the search stage itself multi-objective so that a Pareto frontier of already trained subnetworks is obtained in one run (Ito et al., 2023). The same logic appears in adversarial training, quantized LLM fine-tuning, memory-optimized supernets, and multimodal instruction tuning, where one reusable model or selector is intended to serve multiple hardware or data regimes without retraining (Wang et al., 2020, Yi et al., 2024, Girard et al., 2024, Dong et al., 26 May 2026).

A third meaning is one-pass estimation over a large output space. In goal-conditioned reinforcement learning, LEO replaces naive all-goals relabeling by a network that outputs values and actions for every goal at once, so each transition can update all goals in parallel (Matthews et al., 22 May 2026). In gate-set shadow tomography, a single experimental dataset can be post-processed into standard, interleaved, simultaneous, correlated, and leakage-oriented randomized benchmarking estimates (Silva et al., 21 Oct 2025).

A fourth meaning is conceptual rather than computational. In relational quantum mechanics, “all-at-once” appears as a metaphysical proposal according to which the laws of nature fix the entire distribution of events atemporally; the cited paper argues that this is not required if event sets are treated as non-absolute and observer-relative (Toussaint, 2023).

2. Joint forward models and coupled inverse formulations

In scientific imaging and inverse problems, all-at-once usually denotes a single forward model over all latent variables and all observations. AstroPhot is a particularly explicit example. It models each exposure $e$ and pixel $p$ by

$$M_{e,p}(\theta) = S_{e,p}(\beta_e) + \sum_i [\mathrm{PSF}_e \ast (G_i \circ T_e^{-1})](x_{e,p}; \theta_i) + \sum_j [\mathrm{PSF}_e \ast (\delta \circ T_e^{-1})](x_{e,p} - x_j) f_{j,b(e)},$$

and minimizes a global

$$\chi^2(\theta)=\sum_e \sum_{p\in e} \frac{[D_{e,p}-M_{e,p}(\theta)]^2}{\sigma_{e,p}^2}.$$

The stated benefits are simultaneous deblending, recovery of parameter covariances via Jacobians and Hessians, consistent PSF and background modeling across exposures and bands, and use of native pixels without resampling (Stone et al., 2023).

The inverse-problem literature uses the same idea in a more abstract operator-theoretic form. Rather than first solving $A(x,u)=0$ for $u=S(x)$ and then optimizing over $x$, all-at-once formulations regularize the pair $(x,u)$ directly through a joint forward operator such as $F(x,u)=(A(x,u),C(u))$ (Kaltenbacher, 2016). For dynamic inverse problems on finite horizons, the all-at-once formulation similarly treats state and parameter trajectories together and compares it against reduced methods for Landweber iteration, iteratively regularized Gauss–Newton, and Landweber–Kaczmarz schemes (Kaltenbacher, 2019). In the Bayesian setting, the same reformulation allows priors on both state and parameter and includes model-error and observation-error blocks in a single Gaussian posterior construction (Schlintl et al., 2021).

This suggests a common rationale: the all-at-once formulation is attractive when eliminating intermediate variables would discard useful covariance information, require repeated nonlinear solves, or impose structural assumptions such as a well-behaved parameter-to-state map. In the supplied papers, that rationale appears in astronomy, deterministic regularization theory, and Bayesian inverse problems alike (Stone et al., 2023, Kaltenbacher, 2016, Schlintl et al., 2021).

3. Train-once, search-once, and select-anytime frameworks

A large subset of “all@once” research in machine learning is organized around amortization. OFA$e$0 keeps the Once-for-All supernet paradigm—elastic depth, width, kernel size, and input resolution—but reframes the search stage as a multi-objective optimization that simultaneously optimizes accuracy and resource metrics, yielding a Pareto frontier of non-dominated subnetworks that are already trained and ready to use (Ito et al., 2023). The reported OFA supernet spans a search space of approximately $e$1 subnetworks, and the search itself uses a population of 100 for 1,000 generations (Ito et al., 2023).

Once-for-All Adversarial Training pushes the same principle into robustness. OAT trains a single model $e$2 conditioned on a user-specified knob $e$3, so the standard-accuracy versus robust-accuracy trade-off can be calibrated at test time without retraining; OATS adds a second knob for width, enabling a three-way trade-off among accuracy, robustness, and efficiency (Wang et al., 2020). A closely related deployment logic appears in “One QuantLLM for ALL,” which fine-tunes a quantized LLM supernet once and later extracts subnets under different quantization budgets using per-configuration frozen quantized weights, configuration-specific low-rank adapters, and a non-parametric scheduler over the bit set $e$4 (Yi et al., 2024).

Memory-Optimized Once-For-All retains OFA’s train-once supernet idea but changes the memory geometry of the network itself. Rather than letting an early peak memory layer determine all later stages, MOOFA computes stage-wise channel sizes so that stage peaks remain approximately constant and as high as allowed by the preceding peak, thereby improving memory exploitation and feature diversity under the same peak-memory constraint (Girard et al., 2024). The reported ImageNet results show higher Top-1 accuracy than OFA under peak-memory constraints such as 325k, 350k, 400k, and 800k items, but also substantially higher FLOPs (Girard et al., 2024).

The most literal “train once, select anytime” formulation appears in multimodal instruction tuning. OFA clusters multimodal instructions in a frozen CLIP space, derives pseudo-labels from cluster cores, trains a lightweight selector for only a few epochs, and then selects low-confidence samples as informative examples (Dong et al., 26 May 2026). The selector is trained once on LLaVA-665K and reused without retraining on Vision-Flan-186K; selecting only 15% of the data reaches 98.3% of full-data performance across 10 downstream benchmarks on LLaVA-665K and exceeds full-data training by 10.6% on Vision-Flan-186K (Dong et al., 26 May 2026).

4. Simultaneous learning over structures, actors, and goals

Another major usage of “all@once” concerns the simultaneous treatment of multiple latent structures inside a single model. In structured pruning, the cited work constructs two candidate subnetworks at every iteration—one from layer pruning and one from neuron, filter, or head pruning—and selects the candidate with the highest representation similarity to the parent network, measured by Centered Kernel Alignment (Nascimento et al., 4 Jun 2025). This greedy alternation yields highly sparse models that still preserve predictive ability, with reported milestones of 86.37% FLOPs reduction on ResNet56 and 95.82% on ResNet110 (Nascimento et al., 4 Jun 2025).

In vision–language scene understanding, REACT treats text prompts, video tokens, actor boxes, and group activity labels in one spatiotemporal-semantic pipeline. Its Vision-Language Encoder performs self-attention and bidirectional cross-attention, the Actor Fusion block injects actor-specific box information, and the Action Decoder returns all boxes grounding a text prompt while a classifier reads the same shared representation for group activity recognition (Chappa et al., 2023). The reported results include state-of-the-art performance on Volleyball and JRDB-PAR, with strong degradation when Actor Fusion is removed, which indicates that the “all at once” design is not merely a slogan but a dependency in the architecture (Chappa et al., 2023).

Goal-conditioned reinforcement learning gives this idea an even more literal form. LEO replaces the conventional $e$5 formulation with an all-goals output $e$6, so one forward pass returns values for every goal and action (Matthews et al., 22 May 2026). This removes the $e$7 cost of naive relabeling and yields a reported $e$8 speed-up compared with all-goals relabeling, while significantly outperforming other methods on goal-conditioned Craftax and remaining competitive on continuous-control environments (Matthews et al., 22 May 2026). The paper then uses LEO as a teacher network in Dual LEO to mitigate the late-fusion bottleneck of the base architecture (Matthews et al., 22 May 2026).

This suggests that, in machine learning, “all@once” frequently denotes a move from condition-on-one-instance inputs to tensorized outputs over an entire decision space. The gain is not only computational reuse but also access to cross-instance structure that would otherwise be suppressed by per-instance optimization (Nascimento et al., 4 Jun 2025, Chappa et al., 2023, Matthews et al., 22 May 2026).

5. One-pass computation and one-dataset estimation

In numerical linear algebra and scientific computing, all-at-once often means eliminating intermediate data structures or solving across all time steps simultaneously. For sparse matrix triple products in multigrid, the traditional two-step method forms $e$9 and then $p$0, which incurs extra storage for both $p$1 and $p$2. The all-at-once and merged all-at-once algorithms instead compute $p$3 in a single pass through the input matrices without auxiliary matrices, using hash tables for symbolic and numeric accumulation (Kong, 2019). Reported savings range from about $p$4 memory reduction in structured model problems to about $p$5 in neutron transport, with scalability demonstrated up to 32,768 processor cores and problem sizes exceeding 27 billion unknowns (Kong, 2019).

For evolutionary PDEs, the all-at-once formulation stacks all time levels into one linear system and then applies block circulant preconditioning. The generalized $p$6-circulant preconditioner preserves diagonalizability and identity-plus-low-rank structure, clusters eigenvalues at 1 for sufficiently small $p$7, and yields a GMRES convergence rate independent of the linear-system size when $p$8 is smaller than or comparable to the square root of the time-step size (2002.01108). A related BDF2 all-at-once treatment for Riesz fractional diffusion produces a block triangular Toeplitz system with a low-rank perturbation, then adapts block circulant preconditioners and fast structured Jacobi-matrix solvers to obtain mesh-robust Krylov performance (Gu et al., 2020).

The same one-pass philosophy appears in quantum benchmarking. Gate-set shadow tomography reformulates five common randomized benchmarking techniques under a single protocol based on random gate sequences and sequence correlation functions (Silva et al., 21 Oct 2025). The same dataset supports standard randomized benchmarking, interleaved randomized benchmarking, simultaneous and correlated benchmarking, and a minimal leakage estimate; in addition, simultaneous and correlated variants can reconstruct Pauli terms of correlated noise channels from additional post-processing of only a single experimental dataset, while median-of-means estimators reduce the required experimental sample size (Silva et al., 21 Oct 2025).

6. Limits, trade-offs, and conceptual disputes

A common misconception is that “all-at-once” is uniformly beneficial. The supplied literature does not support that claim. AstroPhot’s full Levenberg–Marquardt mode requires storing a Jacobian of size $p$9, so chunked or iterative fitters are needed for very large scenes (Stone et al., 2023). MOOFA improves memory exploitation and accuracy under peak-memory constraints, but it does so at substantially higher FLOPs (Girard et al., 2024). In LEO, critic updates parallelize efficiently over goals, yet the continuous-control actor update remains $$M_{e,p}(\theta) = S_{e,p}(\beta_e) + \sum_i [\mathrm{PSF}_e \ast (G_i \circ T_e^{-1})](x_{e,p}; \theta_i) + \sum_j [\mathrm{PSF}_e \ast (\delta \circ T_e^{-1})](x_{e,p} - x_j) f_{j,b(e)},$$0, which the paper reports as a throughput reduction of about 70% (Matthews et al., 22 May 2026). In pruning, CKA computation introduces its own overhead, with costs increasing with the number of samples $$M_{e,p}(\theta) = S_{e,p}(\beta_e) + \sum_i [\mathrm{PSF}_e \ast (G_i \circ T_e^{-1})](x_{e,p}; \theta_i) + \sum_j [\mathrm{PSF}_e \ast (\delta \circ T_e^{-1})](x_{e,p} - x_j) f_{j,b(e)},$$1, so sub-sampling and linear CKA are required to keep the method practical (Nascimento et al., 4 Jun 2025).

There is also a conceptual ambiguity in the term itself. In scientific computing, “all-at-once” refers to coupling equations, time slices, or operators. In deployment-oriented machine learning, “once-for-all” refers to amortizing search or adaptation. In randomized benchmarking, it refers to extracting many metrics from one dataset. In relational quantum mechanics, however, it names a metaphysical thesis about the status of events. The cited RQM paper argues against the claim that events must have been created all at once: temporal generation of events is compatible with special relativity if one does not treat the set of events as absolute and observer-independent, and the cross-perspective links postulate applies only along timelike connections (Toussaint, 2023).

This suggests that “all@once” should be read as a family resemblance concept rather than a stable technical term. What persists across domains is not a fixed formalism but a shared reorganization of computation or ontology: replace repeated local updates, isolated fits, or observer-independent event sets with a global structure in which dependencies are handled jointly, and then recover local consequences afterward (Toussaint, 2023, Stone et al., 2023, Matthews et al., 22 May 2026).