Constrained Alignment and Composition

• Constrained Alignment and Composition is a methodology that defines admissible local correspondences using explicit structural, geometric, probabilistic, or categorical constraints before composing them into a global invariant object.
• It employs both hard constraints (e.g., exact routing, logit filtering) and soft constraints (e.g., KL penalties, Lagrangian multipliers) to ensure that the composed structure preserves key properties.
• Empirical evidence across domains such as logic synthesis, vision-language models, and constrained decision making demonstrates high accuracy and stability when local alignments are rigorously constrained prior to composition.

Constrained alignment and composition denotes a family of methods in which local correspondences are not left unconstrained, but are restricted by explicit structural, geometric, probabilistic, optimization, or categorical conditions before being composed into a global object such as a logic circuit, a generated sequence, a merged model, a plan, or a proof. Across the cited literature, the common pattern is to preserve some invariant under composition: exact Boolean structure under discrete compilation, semantic consistency between text and grounded actions, safety geometry during model merging, mechanism fidelity in constrained decisions, or compositional correspondence across modalities (Pavlov, 20 Jan 2026, Andreas et al., 2015, Roy et al., 18 Dec 2025, Qi et al., 16 Jan 2026, Abdollah et al., 2024, Wilson et al., 2021).

1. Conceptual core and scope

At its most abstract, constrained alignment specifies which local pairings, routings, or transformations are admissible; composition then combines those admissible objects while preserving the relevant invariants. In the categorical formulation of “composable constraint encoding,” a lax functor $\mathcal{L}: \mathbf{Con} \to \mathcal{P}[C]$ assigns to each constraint a subset of morphisms, with sequential compatibility enforced by

$$\mathcal{L}(\tau)\circ \mathcal{L}(\lambda)\subseteq \mathcal{L}(\tau\circ\lambda),$$

and relaxations represented by order preservation $$\tau \le \lambda \Rightarrow \mathcal{L}(\tau)\subseteq \mathcal{L}(\lambda)$$. In the formal study of alignments as compositional structures, alignments are likewise treated as structured objects whose columns satisfy order-preservation and non-crossing conditions, and whose restrictions, quotients, and blockwise concatenations remain valid alignments (Wilson et al., 2021, Berkemer et al., 2018).

The same conceptual pattern appears in much more concrete settings. “Differentiable Logic Synthesis” formulates alignment as coefficient selection from a frozen Boolean Fourier dictionary plus Sinkhorn-constrained routing; “Alignment-based compositional semantics for instruction following” aligns sentences to actions and words to grounding-graph nodes under monotonicity and feasibility; “Robust Face Recognition by Constrained Part-based Alignment” regularizes per-part transforms by a tree-structured shape model; “Adaptive Kernel Regression for Constrained Route Alignment” enforces exact waypoints while preserving smoothness; and “Structure of conflict graphs in constraint alignment problems and algorithms” reduces graph alignment to maximum independent set in a conflict graph under one-to-one mapping constraints (Pavlov, 20 Jan 2026, Andreas et al., 2015, Zhang et al., 2015, Du et al., 4 Jan 2026, Alkan et al., 2014).

This breadth indicates that constrained alignment and composition is not tied to a single modality or learning paradigm. It covers symbolic synthesis, probabilistic generation, geometric alignment, structured prediction, category theory, and verification. A plausible implication is that the phrase is best understood as a design principle: admissible local structure is specified first, and only then composed.

2. Formal mechanisms of constraint

One recurring mechanism is convex or manifold-constrained routing. In Hierarchical Spectral Composition, square routing matrices are restricted to the Birkhoff polytope

$$B_k=\{P\in \mathbb{R}^{k\times k}:P\ge 0,\;P\mathbf{1}=\mathbf{1},\;P^T\mathbf{1}=\mathbf{1}\},$$

with projection performed by Sinkhorn scaling. Because pure doubly stochastic routing cannot express Boolean negation, routing is factored as

$$R=P D,\qquad D=\mathrm{diag}(s),\; s\in\{-1,+1\}^k,$$

so that column-sign modulation augments convex combinations with per-output sign flips while retaining the stability properties associated with identity-preserving doubly stochastic manifolds (Pavlov, 20 Jan 2026).

A second mechanism is probabilistic or energy-based constraint composition. In “BERTian Poetics,” masked LLMs are treated as energy-based sequence models over fixed-length sequences, with a standard energy

$$E(x)=-\sum_{i=1}^{l}\log p(x_i\mid x_{\setminus i}),$$

and generation is performed by a Metropolis–Hastings sampler. Constraints are incorporated through a product-of-experts view,

$$P(\mathbf{X}, C_{1:n})=\prod_{i=1}^{n} P(C_i)\cdot P(\mathbf{X}\mid C_{1:n}),$$

and are operationalized by setting forbidden-token logits to $-\infty$ at every sampling step. In the diffusion setting, constrained alignment and composition are also expressed by KL-based optimization: alignment minimizes $D_{\rm KL}(p\|q)$ subject to reward constraints, while product composition minimizes the maximum KL deviation from multiple pretrained models and yields the weighted product-of-experts form

$$q_{\rm AND}(x;\lambda)=\frac{1}{Z_{\rm AND}(\lambda)}\prod_{i=1}^{m} q_i(x)^{\lambda_i}.$$

These formulations make composition explicit at the distribution level rather than only at the parameter level (Akiki et al., 2021, Khalafi et al., 26 Aug 2025).

A third mechanism is geometry-aware protection of sensitive subspaces. In AlignMerge, the merge displacement $$\mathcal{L}(\tau)\circ \mathcal{L}(\lambda)\subseteq \mathcal{L}(\tau\circ\lambda),$$0 is optimized with

$$\mathcal{L}(\tau)\circ \mathcal{L}(\lambda)\subseteq \mathcal{L}(\tau\circ\lambda),$$1

where $$\mathcal{L}(\tau)\circ \mathcal{L}(\lambda)\subseteq \mathcal{L}(\tau\circ\lambda),$$2 keeps the merge close to experts in Fisher-Rao geometry, $$\mathcal{L}(\tau)\circ \mathcal{L}(\lambda)\subseteq \mathcal{L}(\tau\circ\lambda),$$3 penalizes motion along alignment-sensitive directions defined by an alignment Fisher eigenspace, and $$\mathcal{L}(\tau)\circ \mathcal{L}(\lambda)\subseteq \mathcal{L}(\tau\circ\lambda),$$4 imposes a soft alignment budget instantiated with AQI. In XChoice, by contrast, constraints enter through a Lagrangian multiplier in a constrained utility model for time allocation:

$$\mathcal{L}(\tau)\circ \mathcal{L}(\lambda)\subseteq \mathcal{L}(\tau\circ\lambda),$$5

with the shadow price $$\mathcal{L}(\tau)\circ \mathcal{L}(\lambda)\subseteq \mathcal{L}(\tau\circ\lambda),$$6 capturing constraint sensitivity and the share ratios encoding implied trade-offs (Roy et al., 18 Dec 2025, Qi et al., 16 Jan 2026).

These examples illustrate a basic divide. Some works use hard constraints: forbidden logits set to $$\mathcal{L}(\tau)\circ \mathcal{L}(\lambda)\subseteq \mathcal{L}(\tau\circ\lambda),$$7, routing projected into $$\mathcal{L}(\tau)\circ \mathcal{L}(\lambda)\subseteq \mathcal{L}(\tau\circ\lambda),$$8, hard gradient masks, exact waypoint equations, or matching constraints. Others use soft constraints: KL penalties, Lagrangian multipliers, AQI budgets, or quadratic waypoint penalties. The distinction is methodological rather than taxonomic; several systems mix both.

3. Optimization patterns and training procedures

In differentiable logic synthesis, optimization proceeds in phases that explicitly separate continuous relaxation from discrete compilation. Spectral selection is trained with hinge or logistic loss, $$\mathcal{L}(\tau)\circ \mathcal{L}(\lambda)\subseteq \mathcal{L}(\tau\circ\lambda),$$9 sparsity, and the ternary attractor regularizer

$$\tau \le \lambda \Rightarrow \mathcal{L}(\tau)\subseteq \mathcal{L}(\lambda)$$0

while routing logits are exponentiated and Sinkhorn-normalized, signs use a $$\tau \le \lambda \Rightarrow \mathcal{L}(\tau)\subseteq \mathcal{L}(\lambda)$$1 relaxation with annealed $$\tau \le \lambda \Rightarrow \mathcal{L}(\tau)\subseteq \mathcal{L}(\lambda)$$2, and identity preservation is induced by $$\tau \le \lambda \Rightarrow \mathcal{L}(\tau)\subseteq \mathcal{L}(\lambda)$$3. When gradient descent alone fails in larger ternary spaces, the method switches to exhaustive enumeration for $$\tau \le \lambda \Rightarrow \mathcal{L}(\tau)\subseteq \mathcal{L}(\lambda)$$4 and to spectral synthesis plus MCMC with parallel tempering for $$\tau \le \lambda \Rightarrow \mathcal{L}(\tau)\subseteq \mathcal{L}(\lambda)$$5 (Pavlov, 20 Jan 2026).

Several LLM-alignment methods adopt explicit primal-dual or dual-based optimization. “Primal-Dual Direct Preference Optimization for Constrained LLM Alignment” first trains a reward-aware policy via standard DPO, then optimizes a rearranged Lagrangian DPO objective on cost preference data while updating the multiplier by projected dual ascent,

$$\tau \le \lambda \Rightarrow \mathcal{L}(\tau)\subseteq \mathcal{L}(\lambda)$$6

“Alignment of LLMs with constrained learning” alternates Lagrangian maximization in parameter space with dual descent on utility constraints, using the closed-form tilted policy in distribution space as the theoretical reference. Both works treat constrained alignment as maximizing a primary objective while keeping secondary utilities within specified budgets (Du et al., 7 Oct 2025, Zhang et al., 26 May 2025).

Other methods use selective preservation rather than global trust regions. FlipGuard detects update regression through a degradation indicator $$\tau \le \lambda \Rightarrow \mathcal{L}(\tau)\subseteq \mathcal{L}(\lambda)$$7 and adds a conditional congruence penalty,

$$\tau \le \lambda \Rightarrow \mathcal{L}(\tau)\subseteq \mathcal{L}(\lambda)$$8

which reduces to a focal cross-entropy term on negatively flipped examples. Alignment-Aware Probe Pruning uses a KL-based risk-aware gate to decide when to reserve alignment-critical channels before filling the remaining budget by blended probe scores. Constrained Knowledge Unlearning first scores MLP neurons with SNIP-style sensitivity, selects the top $$\tau \le \lambda \Rightarrow \mathcal{L}(\tau)\subseteq \mathcal{L}(\lambda)$$9 as knowledge-related neurons, and then prunes their gradients during gradient-ascent unlearning,

$$B_k=\{P\in \mathbb{R}^{k\times k}:P\ge 0,\;P\mathbf{1}=\mathbf{1},\;P^T\mathbf{1}=\mathbf{1}\},$$0

Here the optimization variable is not just the model; it is the admissible update subspace (Zhu et al., 2024, Patel et al., 9 Nov 2025, Shi et al., 24 May 2025).

A common misconception is that constrained alignment is always implemented as a post-hoc penalty on a preexisting learner. The literature instead includes exact dynamic programs, conflict-graph reductions to maximum independent set, monotone CRF-style inference, Metropolis–Hastings samplers with hard filtering, and semantics-preserving program rewrites via Kleene algebra with tests (Alkan et al., 2014, Andreas et al., 2015, Akiki et al., 2021, Nagasamudram et al., 2023).

4. Representative instantiations across domains

The literature instantiates constrained alignment and composition in markedly different technical forms.

Domain	Alignment object	Constraint/composition mechanism
Logic synthesis (Pavlov, 20 Jan 2026)	spectral coefficients and routing columns	Walsh–Hadamard basis selection, Sinkhorn-constrained routing, column-sign modulation
Instruction following (Andreas et al., 2015)	sentences↔actions and words↔grounding-graph nodes	monotonic sequence alignment, grounding graphs, globally normalized CRF
Vision-LLMs (Abdollah et al., 2024)	entity nodes and relation edges across image and text	node-to-node and edge-to-edge matching, directional FGM, per-level InfoNCE
LLM merging (Roy et al., 18 Dec 2025)	parameter displacement relative to aligned anchor	Fisher-Rao barycenter, alignment-subspace shield, AQI soft budget
Constrained decision making (Qi et al., 16 Jan 2026)	human and LLM mechanism parameters	fixed time budget, Lagrangian multiplier, parameter-vector comparison
Diffusion composition (Khalafi et al., 26 Aug 2025)	pretrained diffusion distributions	reverse-KL constrained product composition, forward-KL mixture composition

Beyond these cases, face recognition treats a face as a piece-wise planar object composed of multiple parts, each with its own 2-D similarity transform, while a tree-structured shape constraint regularizes the relative placement of parts. Route alignment uses adaptive waypoint weights in Nadaraya–Watson regression to decouple global smoothing from local constraint satisfaction. Graph alignment restricts admissible vertex mappings through a similarity graph and studies edge conservation through forbidden subgraphs in the induced conflict graph (Zhang et al., 2015, Du et al., 4 Jan 2026, Alkan et al., 2014).

The same theme also appears in formal reasoning. “Alignment complete relational Hoare logics for some and all” encodes lockstep, left-only, right-only, and joint moves using alignment automata $$B_k=\{P\in \mathbb{R}^{k\times k}:P\ge 0,\;P\mathbf{1}=\mathbf{1},\;P^T\mathbf{1}=\mathbf{1}\},$$1, and filtered automata $$B_k=\{P\in \mathbb{R}^{k\times k}:P\ge 0,\;P\mathbf{1}=\mathbf{1},\;P^T\mathbf{1}=\mathbf{1}\},$$2 for $$B_k=\{P\in \mathbb{R}^{k\times k}:P\ge 0,\;P\mathbf{1}=\mathbf{1},\;P^T\mathbf{1}=\mathbf{1}\},$$3 properties. The associated relational logics are shown alignment complete for their automata classes, with semantics-preserving rewrites based on Kleene algebra with tests enabling programs to be transformed into automaton normal form before relational composition (Nagasamudram et al., 2023).

This diversity shows that composition need not mean neural module composition. It can mean composing proof obligations, blocks of an alignment, distributions, graph matches, control-flow segments, or mechanistic constraints.

5. Empirical and theoretical evidence

In logic synthesis, the reported progression is unusually sharp. For $$B_k=\{P\in \mathbb{R}^{k\times k}:P\ge 0,\;P\mathbf{1}=\mathbf{1},\;P^T\mathbf{1}=\mathbf{1}\},$$4, gradient descent achieves “100% accuracy with zero routing drift and zero-loss quantization” on “all 16 operations”; “column-sign modulation alone ($$B_k=\{P\in \mathbb{R}^{k\times k}:P\ge 0,\;P\mathbf{1}=\mathbf{1},\;P^T\mathbf{1}=\mathbf{1}\},$$5, learn $$B_k=\{P\in \mathbb{R}^{k\times k}:P\ge 0,\;P\mathbf{1}=\mathbf{1},\;P^T\mathbf{1}=\mathbf{1}\},$$6) suffices to recover negations exactly”; and “pure m-style routing without signs caps at 75% (12/16 ops).” For $$B_k=\{P\in \mathbb{R}^{k\times k}:P\ge 0,\;P\mathbf{1}=\mathbf{1},\;P^T\mathbf{1}=\mathbf{1}\},$$7, “pure gradient descent yields ~76% on average,” but “exhaustive enumeration over $$B_k=\{P\in \mathbb{R}^{k\times k}:P\ge 0,\;P\mathbf{1}=\mathbf{1},\;P^T\mathbf{1}=\mathbf{1}\},$$8 configurations finds optimal masks for all 10 operations (100% accuracy), with mean sparsity 39%.” For $$B_k=\{P\in \mathbb{R}^{k\times k}:P\ge 0,\;P\mathbf{1}=\mathbf{1},\;P^T\mathbf{1}=\mathbf{1}\},$$9, “spectral synthesis (exact WHT + ternary thresholding + MCMC) achieves 100% for all 10 operations, mean sparsity 36%,” with “single-cycle combinational logic inference at 10,959 MOps/s (JAX/GPU, RTX 5060)” (Pavlov, 20 Jan 2026).

In alignment-preserving model merging, the empirical comparison is framed directly in terms of alignment geometry. On LLaMA-3 8B, “Naive averaging” yields “AQI 0.59; tox-mean 0.098; tox-rate 22.7%; budget violations 27.6%,” while “AlignMerge (ours)” yields “AQI 0.77; tox-mean 0.038; tox-rate 8.3%; LLM-judge alignment 8.4; budget violations 3.2%; $$R=P D,\qquad D=\mathrm{diag}(s),\; s\in\{-1,+1\}^k,$$0 drift$$R=P D,\qquad D=\mathrm{diag}(s),\; s\in\{-1,+1\}^k,$$1 0.16; $$R=P D,\qquad D=\mathrm{diag}(s),\; s\in\{-1,+1\}^k,$$2 0.06.” The ablations disentangle the roles of the components: “w/o $$R=P D,\qquad D=\mathrm{diag}(s),\; s\in\{-1,+1\}^k,$$3: AQI 0.70; tox-mean 0.061; tox-rate 14.7%; drift increases (0.24), budget violations 14.9%,” whereas “w/o $$R=P D,\qquad D=\mathrm{diag}(s),\; s\in\{-1,+1\}^k,$$4: AQI 0.71; tox-mean 0.058; tox-rate 13.9%; geometry preserved but systematic budget violations 16.8%” (Roy et al., 18 Dec 2025).

In multimodal retrieval and compositional understanding, ComAlign reports consistent gains from level-preserving alignment. For “CLIP ViT-B/32 + ComAlign,” on MSCOCO “I2T R@1 improves from 50.00 to 55.60 (+5.60%), T2I R@1 from 30.35 to 36.62 (+6.27%),” while on ARO VG-Attribution the same model improves “from 61.05 to 66.60 (+5.55)” and on SVO-Probes “from 67.63 to 70.07 (+2.44).” The paper also records that relation directionality remains harder, since “CLIP ViT-L/14 shows a small decrease in VG-Relation (60.98 to 59.53), but overall compositional metrics improve” (Abdollah et al., 2024).

In mechanism-based evaluation of constrained choice, XChoice reports heterogeneity that would be invisible under outcome-only metrics. Model-level divergence is summarized by $$R=P D,\qquad D=\mathrm{diag}(s),\; s\in\{-1,+1\}^k,$$5, where “Claude-3.7: 0.094 (13.1% of total gap)” is the best among the listed models, and feature-level deviations $$R=P D,\qquad D=\mathrm{diag}(s),\; s\in\{-1,+1\}^k,$$6 are largest for “Race—Pacific Islander (0.374; 26.0%), Native American (0.223; 15.6%), Black (0.194; 13.5%), Spouse Present (0.171; 11.9%).” Its invariance analysis shows that under “Race mix shift,” XChoice yields “MAD 0.0374 vs 0.4716; RelL2 0.1487 vs 0.7522; 1−CosSim 0.0111 vs 0.1881 (XChoice vs OLS),” which the paper interprets as mechanism stability under mild perturbations (Qi et al., 16 Jan 2026).

Safety-preserving pruning provides a different form of evidence. For “LLaMA-2-7B-chat” at “fixed prune ratio $$R=P D,\qquad D=\mathrm{diag}(s),\; s\in\{-1,+1\}^k,$$7,” “AAPP 0.57 vs. PP 0.38 vs. Random 0.32,” and on alignment accuracy “PP F1=0.645, Acc=0.624, FAR=0.313; AAPP F1=0.760, Acc=0.741, FAR=0.254.” The same paper notes that “To reach refusal rate 0.5, AAPP needs 10.3 GFLOPs/token” (Patel et al., 9 Nov 2025).

These results do not establish a single universal metric for constrained alignment and composition. Instead, each domain evaluates whether the imposed constraint survives composition: exact truth tables, preserved AQI budgets, compositional retrieval, stable mechanism parameters, or refusal-critical circuits.

6. Limitations, misconceptions, and future directions

A central limitation is scalability of discrete or combinatorial search. In logic synthesis, the paper states that “pure gradient descent struggles in the discrete ternary space ($$R=P D,\qquad D=\mathrm{diag}(s),\; s\in\{-1,+1\}^k,$$8 grows exponentially)” and that “global search/refinement (exhaustive for $$R=P D,\qquad D=\mathrm{diag}(s),\; s\in\{-1,+1\}^k,$$9; MCMC for $$E(x)=-\sum_{i=1}^{l}\log p(x_i\mid x_{\setminus i}),$$0) becomes necessary” (Pavlov, 20 Jan 2026). In graph alignment, hardness remains fundamental even though conflict-graph structure yields improved approximation and fixed-parameter results (Alkan et al., 2014).

Another recurring limitation is that local constraint mechanisms do not automatically solve global structure. In masked-language-model generation, “global constraints (meter, syntax) are more complex to encode as simple logit filters and may need additional checking or soft penalties.” In ComAlign, the method “fails to comprehend the direction of relationships between objects” and “does not fully use the entire graph structure—only node/edge matching is enforced.” These are not minor caveats: they mark the boundary between local admissibility and full structural semantics (Akiki et al., 2021, Abdollah et al., 2024).

A further misconception is to treat alignment as preserved whenever utility is preserved. Several papers explicitly reject that identification. AlignMerge argues that standard merge schemes “can preserve loss while quietly destroying alignment”; FlipGuard shows that post-update models may improve overall while exhibiting “update regression”; and AAPP shows that dynamic pruning can retain efficiency while pruning away refusal-critical circuits if the gating mechanism fails. In each case, the lesson is that alignment must be enforced as an invariant or monitored through mechanism-level quantities, not inferred from aggregate performance alone (Roy et al., 18 Dec 2025, Zhu et al., 2024, Patel et al., 9 Nov 2025).

The current literature also highlights dependence on the chosen constraint proxy. AlignMerge notes “Local geometry assumptions,” “Low-rank proxy,” and “Metric dependence”; XChoice warns about “Structural misspecification risk,” “Prompt dependence,” and the need not to over-interpret subgroup diagnostics as normative; CKU acknowledges “Incomplete unlearning and residual leakage” and the absence of formal convergence or robustness guarantees (Roy et al., 18 Dec 2025, Qi et al., 16 Jan 2026, Shi et al., 24 May 2025).

A plausible implication is that future work will continue to hybridize methods rather than converge on a single recipe. The cited papers already point in that direction: spectral synthesis plus MCMC, dual-based optimization plus preference learning, reward alignment plus closeness constraints, and category-theoretic composition plus task-specific structure. Across domains, constrained alignment and composition appears most effective when three elements are jointly specified: the admissible local correspondences, the operator that composes them, and the invariant that must survive the composition.