d2-AnyOrder: Unordered Modeling & Analysis

• d2-AnyOrder is an order-agnostic framework enabling any-order decoding in masked diffusion language models, delivering unbiased, efficient likelihood estimation.
• It facilitates the discovery of implicit order dependencies in relational databases via SAT-based NP-complete formulations, doubling useful order constraints in data profiling.
• In unordered XML, d2-AnyOrder underpins schema formalisms like DMS/MS, ensuring tractable, expressive validation and optimization of multiset content models.

d2-AnyOrder is a term with distinct definitions and technical import in three research domains: masked diffusion LLMs for reasoning tasks, database theory for discovering implicit order dependencies, and unordered (any-order) XML schema formalisms. In all contexts, "AnyOrder" denotes a methodology or formalism accommodating arbitrary or implicit combinatorial arrangements, with precise technical semantics driven by the underlying application. The dominant research threads are (1) efficient, exact trajectory likelihood estimation for masked diffusion LMs via any-order decoding (Wang et al., 25 Sep 2025); (2) discovery of purely implicit domain orderings contingent on data, under the most general I/I order dependency scenario (Karegar et al., 2020); and (3) multiplicity-only schema specification and tractable analysis for unordered XML (Boneva et al., 2013).

1. Any-Order Decoding and d2-AnyOrder in Masked Diffusion LLMs

In masked diffusion LLMs (MDLMs) for generative text modeling, d2-AnyOrder denotes an algorithmic framework that achieves exact estimation of the likelihood of a sampling trajectory under the assumption that the model admits any-order decoding (Wang et al., 25 Sep 2025). This decoding pattern allows the model to iteratively unmask tokens in arbitrary orders, subject to two essential constraints within the transformer attention architecture: independent masks (mask tokens attend only to themselves and already unmasked tokens) and order causality (a token unmasked at step $t$ attends only to itself and tokens unmasked at steps $<t$). These constraints guarantee that the model's joint likelihood over output sequences factorizes as an any-order autoregressive model (AO-ARM):

$$\pi(\mathbf{x}_{0:T}^{1:L}) = \prod_{l=1}^L \pi(x_0^{\sigma(l)} \mid x_0^{\sigma(1)}, \dots, x_0^{\sigma(l-1)})$$

Here, $\sigma$ is the permutation specifying the unmasking order.

The d2-AnyOrder algorithm evaluates this likelihood exactly with a single transformer pass: the input concatenates the final unmasked tokens and their mask-paired counterparts, positional encodings are shared, and the attention mask implements the stated causal structure. For models trained to enforce any-order mask independence and causality (e.g., Esa-LM, certain finetuned LLaDA variants), d2-AnyOrder provides an unbiased, low-variance estimator for RL policy gradients, outstripping earlier approximate estimators like diffu-GRPO and d2-StepMerge. Empirically, this yields improved performance on logical and mathematical reasoning tasks, such as GSM8K and MATH500 (Wang et al., 25 Sep 2025).

2. d2-AnyOrder for Implicit Order Dependency Discovery in Relational Data

In the domain of database theory, d2-AnyOrder refers to the most general case of discovering order dependencies in relational data, where both sides of the dependency are implicit and unconditional (I/I) (Karegar et al., 2020). Given a relation $r$ and partition context $X$, the goal is to identify partial orders $(A^*,B^*)$ such that $r \models \ordersCtxSet{X}{A^*}{B^*}$—that is, for every group induced by $X$, $A^*$ and $<t$0 are implicit domain orders making $<t$1 order $<t$2. The challenge is to discover the strongest such (uniform across all groups) ordering.

The complexity of validating a d2-AnyOrder dependency is NP-complete, established by reduction from Not-All-Equal 3SAT via the chain polarity problem. Validation is attacked by reducing the problem to SAT: orderings are encoded as propositional variables subject to antisymmetry, no-swap, and transitivity constraints. Solving yields the most informative partial orders, further scored by a "pairwise interestingness" function (fraction of value-pairs ordered).

Algorithmically, d2-AnyOrder employs a level-wise lattice traversal over candidate attribute pairs, functional dependency checks, per-group bipartite graph construction, and ultimately a SAT solver for the unconditional I/I case.

Practical advances include doubling the number of useful order constraints in large data profiling settings and improving performance in query optimization (for example, enabling index reuse and reducing runtimes by up to 30%) and summary-based data mining applications (information-maximizing summaries with 60% higher content using derived orders) (Karegar et al., 2020).

3. Expressive Schema Specification for Unordered XML: AnyOrder Content Models

AnyOrder is also realized in unordered XML through the formalism of Disjunctive Multiplicity Schemas (DMS) and their restriction, Multiplicity Schemas (MS) (Boneva et al., 2013). These schema languages specify the allowed multiplicities of XML element children independently of sibling order, modeling tree-structured data with arbitrary child arrangements.

In DMS, rules are formulated as disjunctive multiplicity expressions (DME), succinctly encoding unordered collections (multisets) of symbols with cardinality constraints. For instance,

• $<t$3 with all $<t$4 appearing uniquely across disjunctions;
• Each $<t$5 governed by per-symbol multiplicities (*, +, ?, 1, 0).

Membership (validation), satisfiability, and containment checking for schemas and queries are tractable (PTIME) for key classes; PTIME for validation and containment, NP-complete or EXPTIME-complete for some complex query tasks in DMS, and PTIME/coNP-complete for many tasks in MS.

The essential significance is that, using DMS/MS, specification and tractable analysis of unordered (AnyOrder) XML content models become practical without sacrificing the expressive power of more complex, order-sensitive regular tree grammars (Boneva et al., 2013).

4. Algorithmic and Complexity Properties

Setting	Core Problem	Complexity
d2-AnyOrder in MDLMs	Exact trajectory likelihood	Single transformer pass
d2-AnyOrder in Databases	Implicit OD discovery (I/I)	NP-complete (SAT-based)
AnyOrder XML (DMS/MS)	Schema/validation/query	PTIME/coNP/EXPTIME

The d2-AnyOrder estimator for MDLMs achieves $<t$6 computational cost (one pass through a length-$<t$7 sequence, $<t$8 per-token cost), whereas naive reverse-diffusion evaluation costs $<t$9 with $$\pi(\mathbf{x}_{0:T}^{1:L}) = \prod_{l=1}^L \pi(x_0^{\sigma(l)} \mid x_0^{\sigma(1)}, \dots, x_0^{\sigma(l-1)})$$0. For database order dependency discovery, the unconditional I/I case is NP-complete but tractable in practice via SAT encoding. Unordered XML schemas (DMS/MS) allow for static analysis tasks to be performed in polynomial or coNP/EXPTIME, depending on the problem class (Wang et al., 25 Sep 2025, Karegar et al., 2020, Boneva et al., 2013).

5. Model and System Compatibility

d2-AnyOrder in masked diffusion LMs is unbiased only for models trained to satisfy the attention constraints needed for any-order decoding—those that guarantee mask independence and order causality during pretraining (for example, Esa-LM or models finetuned via AO-ARM objectives). For standard MDLMs without these guarantees, the estimator can yield arbitrarily incorrect likelihoods, necessitating either model modification (AO finetuning) or fallback to approximate estimators (such as d2-StepMerge) (Wang et al., 25 Sep 2025).

In data profiling, d2-AnyOrder's effectiveness is determined by the absence of strong explicit FDs and the presence of complex, distributed implicit ordering phenomena, which it can uncover due to its unconstrained approach. In XML schema analysis, DMS/MS schemes are applicable whenever unordered sibling content models with multiplicity constraints suffice to capture the required data semantics (Karegar et al., 2020, Boneva et al., 2013).

6. Empirical and Practical Impact

Empirical studies of d2-AnyOrder within RL-optimized MDLMs demonstrate state-of-the-art performance on logical and mathematical reasoning tasks. For instance, on GSM8K and MATH500 benchmarks, d2-AnyOrder improves test accuracy by up to 10 points over baseline diffusion-based RL learners, due to low-variance, unbiased importance weights in policy gradient optimization. In database and data mining contexts, the d2-AnyOrder approach substantially expands the set of order constraints available for profiling, enables query optimizations (with observed 30% runtime reductions in industrial settings), and enhances the informativeness of data summaries on real-world datasets (Wang et al., 25 Sep 2025, Karegar et al., 2020).

7. Synthesis and Thematic Connections

Across these domains, d2-AnyOrder and AnyOrder formalism articulate a common paradigm: computational frameworks that allow operations, inference, discovery, or specification to proceed without imposed or fixed sequencing. This is realized through algorithmic enforcement or exploitation of commutativity, mutual independence, or the detection of order structure purely from observed data or model constraints. In masked diffusion models, this enables exact, efficient trajectory likelihoods for powerful RL; in databases, the extraction of deeply latent patterns augmenting classical dependencies; and in XML, schema expressivity and tractability for unordered, highly variable data. The underlying mathematical and algorithmic techniques—permutation-invariant factorization, SAT-based orientation inference, multiset and commutative grammar analysis—establish d2-AnyOrder as a central method for order-agnostic reasoning and specification in contemporary computational research (Wang et al., 25 Sep 2025, Karegar et al., 2020, Boneva et al., 2013).