Inverse Logic-Constraint Learning

• Inverse Logic-Constraint Learning is a framework that infers constraints from observed decisions to recover model structure for safe and optimal control.
• It employs diverse formulations, including mixed-integer programming, constrained MDPs, and temporal logics, to capture both Markovian and non-Markovian behaviors.
• The approach underpins applications in scheduling, robotics, and safe control, using algorithms like MILP solves, SMT encoding, and genetic mining.

Inverse Logic-Constraint Learning (ILCL) denotes a class of inverse problems in which constraints are inferred from observed behavior, decisions, or trajectories rather than being specified a priori. Across recent work, the learned object may be a thresholded constraint template in mixed-integer optimization, a conjunction of state/action predicates in an MDP, a formula in the $GF$ fragment of Linear Temporal Logic (LTL), or a parameterized truncated Linear Temporal Logic (TLTL) specification over finite trajectories. In several formulations, ILCL is not only a constraint-learning problem: the learned constraint is subsequently paired with objective estimation, constrained policy optimization, or automaton-based planning, so that the recovered model reproduces observed feasible or expert-like behavior (Kitaoka, 6 Oct 2025, Scobee et al., 2019, Afzal et al., 2021, Cho et al., 15 Jul 2025, Qadri et al., 26 Jan 2025).

1. Conceptual scope and problem classes

ILCL appears in at least five closely related technical settings. In inverse mixed-integer programming, the problem is to learn logic-type constraints and then objective coefficients from observed optimal decisions. In inverse reinforcement learning and constrained MDPs, the problem is to infer hard constraints that, together with a nominal reward, explain demonstrations. In temporal-specification learning, the problem is to infer a non-Markovian logical formula that separates desirable and undesirable traces or that all demonstrations satisfy while learned policies are penalized for violating it. In safe-control formulations based on safe demonstrations, ILCL is also called Inverse Constraint Learning (ICL), and the learned constraint is interpreted through reachability theory rather than purely as a symbolic specification (Kitaoka, 6 Oct 2025, Scobee et al., 2019, Afzal et al., 2021, Cho et al., 15 Jul 2025, Qadri et al., 26 Jan 2025).

A useful way to organize the field is by the mathematical form of the constraint.

Setting	Constraint representation	Learned object
Inverse mixed-integer programming	$g(x,\phi,s)\le 0$ with threshold parameters $\phi$	$\phi^{sup}$, then $\theta$
MDP constraint inference	Boolean predicates $\phi_k(s,a)\in\{0,1\}$	Constraint set $C$
Non-Markovian IRL	$GF$-fragment LTL in negation normal form	Formula $\varphi$
Temporally constrained demonstrations	Parametric TLTL over finite traces	Constraint $\varphi$ and policy $g(x,\phi,s)\le 0$0
Safe demonstrations under dynamics	Classifier-defined unsafe set	A set shown to equal $g(x,\phi,s)\le 0$1 under assumptions

This suggests that ILCL is better understood as a family of inverse modeling paradigms united by a common aim: recovering a declarative feasible set, safety rule, or temporal specification from behavior. The principal differences lie in whether constraints are Markovian or non-Markovian, symbolic or threshold-parametric, and whether they are learned jointly with rewards or in a staged pipeline.

2. Formal representations and semantics

In the mixed-integer formulation, the forward optimization problem is

$g(x,\phi,s)\le 0$2

where $g(x,\phi,s)\le 0$3 contains continuous and integer variables, $g(x,\phi,s)\le 0$4 is the context, $g(x,\phi,s)\le 0$5, and the constraint family is parameterized by thresholds $g(x,\phi,s)\le 0$6. A canonical template is

$g(x,\phi,s)\le 0$7

Here $g(x,\phi,s)\le 0$8 enforces fixed constraints, while $g(x,\phi,s)\le 0$9 and $\phi$0 define learnable upper- and lower-threshold relations. The inverse task is to find $\phi$1 so that observed decisions are optimal or nearly optimal for the forward problem.

In constrained MDP formulations, hard constraints are Boolean predicates over state-action pairs. For each predicate $\phi$2, a trajectory satisfies the constraint if $\phi$3 for all $\phi$4, which can be written as the safety LTL formula $\phi$5. The feasible trajectory set is

$\phi$6

Under MaxEnt IRL with nominal reward $\phi$7, the constrained trajectory model becomes

$\phi$8

In the LTL-based non-Markovian setting, the learner searches within the $\phi$9 fragment in negation normal form: $\phi^{sup}$0 The key novelty is a quantitative valuation $\phi^{sup}$1 on finite words $\phi^{sup}$2, using $\phi^{sup}$3 as a temporal discount and $\phi^{sup}$4 as a nesting discount. For example,

$\phi^{sup}$5

Positive traces must satisfy the learned formula, negative traces must not, and the valuation ranks consistent formulas by simplicity and temporal economy.

In TLTL-based ILCL, formulas are evaluated on finite traces through robust semantics. Atomic predicates take the form $\phi^{sup}$6, and the sign of the robustness score $\phi^{sup}$7 determines satisfaction: $\phi^{sup}$8 Temporal operators include $\phi^{sup}$9, with standard min-max finite-horizon robustness recursions. This makes TLTL especially suited to episodic robotic tasks, because satisfaction is defined on finite trajectories and the magnitude of violation can be used as a graded constraint cost.

3. Learning paradigms and algorithms

The mixed-integer two-stage method separates constraint learning from objective learning. Stage 1 learns the largest thresholds consistent with feasibility of all observed solutions: $\theta$0 For finite $\theta$1, this reduces to

$\theta$2

so the learned constraint is the component-wise meet of single-state thresholds. Stage 2 fixes $\theta$3 and optimizes $\theta$4 by projected stochastic gradient descent on the suboptimality loss, using $\theta$5 forward MILP solves per iteration. In the affine threshold case, Stage 1 is closed-form and requires no MILP solve.

Maximum-likelihood constraint inference in MDPs uses a different mechanism. The log-likelihood objective reduces to maximizing the nominal trajectory mass eliminated by a constraint while preserving all demonstrations. The paper expresses this as choosing $\theta$6 to maximize $\theta$7 subject to $\theta$8. Candidate constraints are atomic state, action, or feature predicates. A feature-accrual history estimates how much nominal probability mass each predicate would eliminate, and a greedy maximum-coverage procedure adds constraints until the decrease in $\theta$9 falls below a threshold.

LTL-based non-Markovian ILCL is formulated as a constraint-system optimization problem over formula syntax trees. One method encodes an unknown $\phi_k(s,a)\in\{0,1\}$0-formula of bounded depth into quantifier-free SMT constraints and maximizes the minimum positive-trace valuation while enforcing zero valuation on negatives. Two related alternatives are optimized pattern matching, where a partial template is given, and hybrid pattern matching, which synthesizes unspecified subtrees. A separate compositional ranking algorithm is incomplete but fast: it enumerates formulas up to a depth bound, prunes with an $\phi_k(s,a)\in\{0,1\}$1-check, computes $\phi_k(s,a)\in\{0,1\}$2, and sorts candidates.

The TLTL formulation introduces a two-player zero-sum game between a constraint player and a policy player. The constraint player, GA-TL-Mining, evolves syntax trees for parameterized TLTL without predefined templates. Its base empirical fitness is

$\phi_k(s,a)\in\{0,1\}$3

and parent selection adds a complexity penalty through

$\phi_k(s,a)\in\{0,1\}$4

The policy player, Logic-CRL, converts the learned formula to a DFA, constructs a product CMDP, and solves a Lagrangian constrained RL problem using SAC-Lag. A dense trajectory-level cost mixes binary violation and clipped negative robustness,

$\phi_k(s,a)\in\{0,1\}$5

and then redistributes this cost to state-action-product-state tuples to address non-Markovian credit assignment.

4. Optimality, identifiability, and generalization

A central issue in ILCL is that rewards and constraints can be confounded. The mixed-integer work addresses this directly: if both $\phi_k(s,a)\in\{0,1\}$6 and $\phi_k(s,a)\in\{0,1\}$7 vary freely, changes in $\phi_k(s,a)\in\{0,1\}$8 alter the feasible region and can obscure objective inference. The two-stage design fixes this by first learning $\phi_k(s,a)\in\{0,1\}$9, then learning $C$0 on a fixed feasible set. The exactness of the approach is tied to the suboptimality loss

$C$1

with the equivalence

$C$2

Under piecewise-linear $C$3 and polyhedral feasible sets, the two-stage procedure solves the inverse problem on finite data for almost every $C$4, and the paper also derives generalization bounds in pseudometric parameter spaces under sub-Gaussian assumptions. In the specialization $C$5, the expected excess suboptimality scales as $C$6 up to problem-dependent constants.

The LTL-based formulation provides a different kind of rigor. Within the bounded-depth $C$7 search space, the SMT encoding is proved complete and sound: if there exists a depth-$C$8 consistent formula, then there exists a satisfying model of the constraint system, and conversely every satisfying model decodes to a consistent formula with valuation variables equal to the intended semantic values. The corollary states valuation equivalence: for any $C$9 and finite trace $GF$0, $GF$1 iff $GF$2.

In MDP likelihood-based inference, identifiability is deliberately biased toward constraints by fixing a nominal reward. The method does not attempt to recover all true constraints; it recovers those that most increase demonstration likelihood by eliminating high-probability nominal behavior that was not observed. This is why low-probability true violations may remain uninferred.

A major conceptual correction is provided by the safe-control analysis of ICL. Under maximum causal entropy, safe demonstrations, a vacuous initial constraint $GF$3, and a perfect logistic classifier separating learner and expert states, the learned unsafe set after one exact outer update equals the backward reachable tube $GF$4, not the failure set $GF$5. The paper’s quantified safe set is

$GF$6

with $GF$7. This result means that ILCL from safe-only demonstrations is dynamics-conditioned: what is recovered depends on the data-collection dynamics, control limits, disturbance set, and horizon.

5. Applications and empirical evidence

The empirical record for ILCL spans optimization, formal-specification mining, robotics, and safe control (Kitaoka, 6 Oct 2025, Afzal et al., 2021, Cho et al., 15 Jul 2025, Scobee et al., 2019, Qadri et al., 26 Jan 2025).

In inverse mixed-integer programming, the principal application is scheduling. The paper studies single-machine scheduling $GF$8 with integer start times $GF$9, binary precedence variables $\varphi$0, and learned precedence permissions $\varphi$1. Experiments on ILPs with up to $\varphi$2 decision variables report a mean training time of $\varphi$3 seconds and a maximum of $\varphi$4 seconds. Constraint learning is essentially instantaneous, because it is computed directly from schedules via lattice operations, while objective learning dominates runtime through repeated forward MILP solves.

In LTL-based non-Markovian IRL, synthetic gridworld experiments use $\varphi$5 environments, trace lengths $\varphi$6–$\varphi$7, and typically $\varphi$8 positive and $\varphi$9 negative traces per formula. Reported mean inverse learning error is $\varphi$0 for Constraint Optimization, $\varphi$1 for Compositional Ranking, and $\varphi$2 for Traces2LTL. The method is also evaluated on Dining Philosophers traces, where template-guided learning discovers formulas such as $\varphi$3, mutual exclusion constraints, and deadlock-freedom patterns, with runtimes of approximately $\varphi$4 s, $\varphi$5 s, and $\varphi$6 s for the listed patterns.

The TLTL-based two-player ILCL is evaluated on four temporally constrained tasks: Navigation, Wiping, and Peg-in-shallow-hole, with two navigation specifications. The paper reports the lowest violation rate (VR) and truncated negative robustness (TR) among baselines on both seen and unseen environments, while maintaining expert-like normalized reward (REW). Example learned constraints remain structurally close to the ground-truth formulas. Across tasks, the mined formulas achieve average VR $\varphi$7 and REW $\varphi$8 on their own constrained policies. The method also transfers to a real-world peg-in-shallow-hole task on a $\varphi$9-DoF Franka Emika Panda arm, with each policy trained for approximately $g(x,\phi,s)\le 0$00 hours or about $g(x,\phi,s)\le 0$01 interaction steps.

Maximum-likelihood constraint inference in MDPs is demonstrated on a synthetic $g(x,\phi,s)\le 0$02 gridworld and on human obstacle avoidance. In the gridworld, the method recovers high-impact feature, action, and state constraints in sequence; decreasing the KL threshold $g(x,\phi,s)\le 0$03 increases false positives, while more demonstrations reduce false positives and final $g(x,\phi,s)\le 0$04. A threshold around $g(x,\phi,s)\le 0$05 is reported as a good trade-off. In the human study with $g(x,\phi,s)\le 0$06 volunteers, obstacle-present demonstrations lead the algorithm to detect the obstacle region as a set of state constraints, making those states unlikely under the inferred model.

In safe control, experiments use a Dubins car-like system with two dynamics models: an agile model with $g(x,\phi,s)\le 0$07, and a non-agile model with $g(x,\phi,s)\le 0$08, both under disturbances $g(x,\phi,s)\le 0$09. With about $g(x,\phi,s)\le 0$10 safe expert trajectories per model, the learned classifier aligns closely with HJ-computed backward reachable tubes. Transfer across dynamics fails in both directions described by the theory: a constraint learned under agile dynamics underestimates unsafe states for the non-agile model, while a constraint learned under non-agile dynamics is safe but overly conservative when applied to the agile model.

6. Limitations, misconceptions, and research directions

Several recurring limitations define the current boundary of ILCL (Kitaoka, 6 Oct 2025, Afzal et al., 2021, Cho et al., 15 Jul 2025, Scobee et al., 2019, Qadri et al., 26 Jan 2025).

A first misconception is that ILCL always recovers the “true” underlying constraint. The safe-control analysis shows that, from safe-only demonstrations, the learned object is a backward reachable tube rather than the failure set itself. A second misconception is that logic constraints remove ambiguity. In practice, underdetermination remains pervasive: multiple $g(x,\phi,s)\le 0$11 pairs can rationalize the same mixed-integer observations, multiple LTL or TLTL formulas can separate the same traces, and nominal reward misspecification in MDPs can shift explanatory burden onto constraints.

Template dependence is another major issue. The mixed-integer formulation requires a lattice-homomorphism template in which each $g(x,\phi,s)\le 0$12 is monotone in a single threshold variable. The MDP likelihood formulation relies on a library of atomic predicates. The $g(x,\phi,s)\le 0$13-fragment method is restricted to Boolean connectives plus $g(x,\phi,s)\le 0$14 and $g(x,\phi,s)\le 0$15, with negations only on literals. TLTL-based ILCL removes predefined templates but still depends on atomic predicate design $g(x,\phi,s)\le 0$16, genetic search heuristics, and finite-depth syntax trees.

Noise and demonstration quality matter differently across formulations. The mixed-integer theory assumes precise observed optima; noisy data can make $g(x,\phi,s)\le 0$17 overly conservative. The LTL method addresses noise partly through margins and quantitative semantics, and synthetic experiments inject extraneous atomic propositions with $g(x,\phi,s)\le 0$18. The TLTL game enforces strict demo satisfaction in mining, which can be brittle under noisy traces. The safe-control theorem assumes safe demonstrations, exact entropy-regularized optimization, and sufficient coverage; imperfect experts and limited state coverage soften the learned boundary.

Computation remains a persistent bottleneck. Mixed-integer objective learning requires repeated MILP solves. SMT-based temporal-logic synthesis is exponential in depth and atomic propositions. Genetic TLTL mining adds dual annealing over formula parameters and nonconvex alternation with constrained RL. Classifier-based safe-control ILCL requires repeated constrained policy learning plus outer-loop discrimination. These costs explain the field’s emphasis on staged pipelines, approximate search, and bounded-depth or template-guided variants.

The research directions stated across the cited works are relatively consistent. They include robust or noise-aware ILCL, richer logics such as full LTL, STL, and probabilistic temporal logics, joint learning of atomic propositions or features with formulas, model selection over logical structure, multi-dynamics aggregation to approximate the true failure set by intersecting learned unsafe sets, scalable high-dimensional backward reachable tube estimation, and extensions from linear or piecewise-linear objectives to nonlinear settings. A plausible implication is that future ILCL systems will combine symbolic structure search, robustness-aware statistical estimation, and downstream constrained planning in a single integrated pipeline, but current methods remain specialized to the representational and computational assumptions of their respective domains.