Solving Stochastic Constraints by Oracle-based Gradient Descent and Interval Arithmetic

Abstract: Stochastic constraints, which incorporate both deterministic parameters and random variables, extend classical deterministic constraints by explicitly accounting for uncertainty. These constraints are increasingly prevalent in data science, artificial intelligence, and bioinformatics; however, solving them requires addressing quantitative satisfaction problems that remain a significant challenge in computer science. In this paper, we propose a novel framework for deciding deterministic parameters that maximize the satisfaction probability. Our approach features a unique synergy between stochastic optimization and symbolic techniques: at the high level, it employs \emph{oracle-based stochastic gradient descent} to identify high-quality parameter candidates, while at the low level, it utilizes \emph{interval arithmetic} to compute rigorously certified lower bounds. This framework produces a sequence of sound and increasingly tight lower bounds for the true maximum satisfaction probability, supported by a high-probability convergence guarantee. We demonstrate the effectiveness and efficiency of our approach through its application to Stochastic Satisfiability Modulo Theories (SSMT) problems and a stochastic trajectory planning task.

• The paper introduces a hybrid framework using oracle-based gradient descent to maximize satisfaction probabilities under stochastic constraints.
• It leverages interval arithmetic for certifying lower bounds, ensuring formal verification of probabilistic constraints in non-convex settings.
• Empirical evaluations demonstrate high-confidence, scalable performance in applications like SMT solving and uncertainty-aware trajectory planning.

Solving Stochastic Constraints via Oracle-Based Gradient Descent and Interval Arithmetic

Problem Setting and Motivation

This work presents a rigorous algorithmic framework for the optimization of satisfaction probabilities in semialgebraic stochastic constraints. The formulation considers constraints $\varphi(\boldsymbol{x}, \boldsymbol{y})$ parameterized by deterministic variables $\boldsymbol{x}$ and random variables $\boldsymbol{y}$ with known distributions. The central computational objective is to determine parameters $\boldsymbol{x}$ that maximize the satisfaction probability $$V_\varphi(\boldsymbol{x}) = \mathbb{P}_{\boldsymbol{y}}[\varphi(\boldsymbol{x}, \boldsymbol{y})]$$, a class of problems fundamental in areas such as data science, formal methods, and probabilistic planning.

Historically, tractable solution methods for stochastic constraints have required restrictive assumptions (e.g., discrete domains, scenario-based approximations, convexity, or specific structure in chance-constrained optimization). This work extends beyond these limitations by directly handling general first-order semialgebraic formulas over continuous domains and distributions. The key challenge is to compute lower bounds for the generally intractable probabilistic satisfiability criteria with formal certification, while scaling to moderately high dimensions.

Algorithmic Contributions

The core contribution is a modular hybrid framework combining stochastic gradient-based global optimization and interval-certified statistical estimation.

1. Oracle-Based Stochastic Optimization:

The paper frames the maximization of satisfaction probability as the minimization of an expected $0$-$1$ loss function, allowing the use of stochastic first-order methods. Since direct evaluation of $V_\varphi(\boldsymbol{x})$ and its gradients is infeasible, two stochastic oracles are defined:

• Zeroth-order oracles estimate $V_\varphi(\boldsymbol{x})$ by empirical sampling.
• First-order oracles estimate the gradient via randomized finite-difference smoothing on the sampled empirical function.

These oracle constructions are shown to satisfy strictly sub-exponential concentration properties, uniformly over the parameter space, strengthening prior work by removing the independence assumptions on oracle errors.

2. Interval Arithmetic for Lower Bound Certification:

For any candidate $\boldsymbol{x}$ (arising from the optimizer), interval constraint propagation and recursive branching subdivide the space of $\boldsymbol{x}$0, separating inner regions (guaranteed to satisfy the constraint) from boundary regions. This yields a lower-bounding function $\boldsymbol{x}$1 that provides an arbitrarily tight, certification-ready under-approximation of $\boldsymbol{x}$2.

3. Integrated Two-Stage Algorithm:

The algorithm alternates multi-start stochastic optimization (using the ALOE line-search framework) and lower-bound certification on the returned candidates. Running $\boldsymbol{x}$3 independent ALOE instances and retaining the best lower bound yields a sound and tight certified lower bound for the true optimum. The approach decouples parameter search from formal verification, allowing aggressive optimization without loss of soundness.

Theoretical Analysis

The work presents detailed complexity analyses and non-asymptotic high-probability convergence guarantees:

• The expected $\boldsymbol{x}$4-$\boldsymbol{x}$5 loss objective is shown to be almost everywhere differentiable and absolutely continuous, justifying the stochastic optimization methodology even in the semialgebraic, non-convex, and non-smooth regime.
• Crucially, the dependency structure of the oracle errors is handled by a novel concentration inequality for martingale-type sums, bypassing the independence requirement common in prior oracle-based analyses.
• The main theorems detail that the probability of certifying a lower bound within $\boldsymbol{x}$6 (verification plus optimization error) of the true satisfaction probability increases exponentially in both the number of independent multi-starts $\boldsymbol{x}$7 and the number of line-search steps $\boldsymbol{x}$8, given minimal regularity (i.e., local Lipschitz conditions around optima).

This positions the method as both formally sound (the bounds are always valid, regardless of stochastic fluctuations) and practically efficient (exponential tail decay in error probability as resource budgets increase).

Empirical Results

Comprehensive experiments are conducted on two benchmarks:

1. Stochastic SMT:

The method solves continuous-domain stochastic exists-forall SMT problems, with each instance involving conjunctions and disjunctions of nonlinear polynomial inequalities over a mixture of existential and universally (randomized) quantified variables. Certified lower bounds are produced with errors matching the prescribed arithmetic precision and robust performance (relative errors $\boldsymbol{x}$9 to $\boldsymbol{y}$0), even extending beyond the proven theoretical assumptions in some pathological cases.

2. Trajectory Planning under Uncertainty:

For multi-step motion planning problems with additive and multiplicative environmental noise and polynomial-Gaussian/non-uniform randomness, the method produces high-confidence optimal plans. Certified lower bounds for satisfaction probabilities routinely exceed 0.99 in complex obstacle-laden environments. The computational scalability is favorable for small to medium instances, with the overall complexity governed more by intrinsic geometric/topological landscape than by raw constraint count.

Implications and Future Directions

The framework advances the state-of-the-art in stochastic constraint satisfaction by:

• Providing formal, certified lower bounds where previously only approximate (or structural-assumption-heavy) methods existed.
• Delivering provable sample complexity and iteration bounds under minimal regularity, particularly for non-convex, non-smooth, indicator-based stochastic objectives.
• Bridging symbolic-numeric verification with scalable, probabilistically robust stochastic optimization.

Practically, the approach can generalize to other domains involving quantified semialgebraic constraints under uncertainty, including hybrid system verification, chance-constrained control, and formal AI planning under distributional ambiguity.

Theoretically, future enhancements can focus on scalable certification techniques for higher dimensions, integration with sum-of-squares relaxations or symbolic reasoning where applicable, and extension to broader classes of constraints (e.g., transcendental, mixed discrete-continuous, or nonlinear hybrid logics).

Conclusion

This work demonstrates a sound and efficient framework for solving stochastic constraint optimization problems with formal guarantees, blending the stochastic approximation power of oracle-based gradient methods and the rigor of interval arithmetic. The approach closes gaps in scaling bounded probabilistic verification to complex, non-convex settings, and its high-probability iteration complexity and validation properties establish a solid foundation for future robust AI and formal verification applications.

Reference: "Solving Stochastic Constraints by Oracle-based Gradient Descent and Interval Arithmetic" (2604.17275)