Memoization of Non-Deterministic Choices

• Memoization of non-deterministic choices is the reuse of a resolved decision to ensure that repeated evaluations yield a consistent outcome.
• It underpins various implementations such as shared suspensions, tabling, and memoized pull-tabbing to optimize computational processes.
• This approach distinguishes call-time from run-time choice, serving as a semantic control parameter that balances efficiency with expressive power.

Memoization of non-deterministic choices is the reuse of a previously resolved choice in place of re-evaluating a non-deterministic computation. Across the literature, the same idea appears as call-time choice in functional logic programming, tabling of multiple answers and continuations, stateful refinement of coinductive choice structures, distribution-wide commitment of observer choices, and memoized streams that stabilize classical choice proofs. In each case, the central issue is the same: whether repeated uses of one non-deterministic source denote one shared decision or many independent recomputations (0903.2205, Abdallah, 2017, Miquey, 2019).

1. Semantic core: shared choice versus fresh choice

In functional logic programming, the canonical distinction is between call-time choice and run-time choice. Under call-time choice, when a non-deterministic expression is passed as an argument, the choice is made once at the call and all uses of that argument share the same result. Operationally, this is implemented via sharing, graph reduction, or suspensions, so it is literally a memoization of a non-deterministic computation at the call site. Under run-time choice, each occurrence may be reduced independently, closer to ordinary rewriting; every use may make its own choice, so no such memoization is enforced. Deterministic programs do not distinguish the two semantics, but with non-determinism run-time choice usually yields a superset of the call-time results (0903.2205).

The difference is semantic rather than merely operational. The palindrome example from functional logic programming requires memoization: if palAux(X) -> X ++ reverse(X) is called with a non-deterministic word, the two occurrences of X must denote the same underlying word. Without sharing, the left and right halves can evolve independently, yielding strings that are not palindromes. Conversely, grammar combinators such as star(letter) require fresh choices: under pure call-time choice, all occurrences of letter created by star share the same non-deterministic choice, so only words consisting of one repeated letter are produced. The localized annotation rt(e) was introduced precisely to control this boundary: the default semantics memoizes non-deterministic choices at call time, while rt(e) says that the marked subexpression should not be shared and should be re-evaluated independently on each use (0903.2205).

The corresponding formal picture is explicit. The language uses constructors, function symbols, variables, and a unary rt marker. rt(e) is desugared by replacing function symbols inside e with annotated variants f^{rt}. In the let-rewriting calculus, a LetIn rule introduces sharing only for calls headed by unannotated function symbols; calls headed by f^{rt} are never hoisted into let, hence are never shared. The absence of let is therefore the formal rendering of “do not memoize this non-deterministic choice” (0903.2205).

A useful synthesis is that memoization of non-deterministic choices is not uniformly desirable. Some computations require a single committed decision to preserve an invariant, while others require repeated independent sampling. The literature therefore treats memoization as a semantic control parameter, not merely as an optimization.

2. Operational mechanisms in implementations

The operational realization of memoized non-determinism is especially explicit in functional logic implementations. In Toy, call-time choice is implemented with suspensions of the form susp(FunctionName, Arguments, Result, EvaluatedFlag). The Result cell stores the memoized value once evaluation occurs; the EvaluatedFlag is on after evaluation and a free variable before evaluation. The run-time annotation is compiled by changing this flag to rt, which means “do not share”: evaluating a suspension with flag rt does not bind Result, so duplicated uses do not reuse any stored outcome. In this sense, the entire extension from call-time choice to mixed call-time/run-time choice is implemented by a local change in the handling of suspension reuse (0903.2205).

Pull-tabbing exposed a different implementation problem. Classical pull-tabbing moves a demanded choice upward by local graph transformations, so f (e_1 ? e_2) becomes (f e_1) ? (f e_2). This preserves flexible search strategies and supports call-time choice when combined with choice identifiers and per-task fingerprints, but shared subexpressions may cause the same logical choice to be pull-tabbed repeatedly. “Memoized Pull-Tabbing” replaces this repeated work by storing, for each graph node n and task i, a task-specific result n.tr(i). Each node also has an owner task ot, and ancestor lookup allows descendant tasks to reuse deterministic results computed earlier. On the reported benchmarks, this removes the catastrophic overhead of repeated pull-tabs: for select100, the first Julia implementation reports 0.18s for memoized pull-tabbing, 111.47s for pure pull-tabbing, and 0.55s for backtracking; for deterministic benchmarks such as nrev and takPeano, the overhead remains small (Hanus et al., 2020).

A monadic implementation in Haskell-like style shows that this machinery is not tied to imperative runtimes. The transformed language uses a monad Curry for non-deterministic computations and an explicit sharing primitive

share :: Curry a -> Curry (Curry a)

whose implementation maintains branchID, parentIDs, and an idSupply. Each shared computation owns an IORef-backed heap keyed by branch identifiers, so a computation is evaluated at most once per branch, while deterministic results can be reused in descendants. The implementation extends this mechanism to functional patterns, free variables, unification, and encapsulated search, and the paper states that, after static and dynamic optimizations for purely functional computations, the resulting system achieves a promising performance that outperforms current compilers for Curry (Hanus et al., 30 Apr 2026).

These implementations make a common point precise: memoization is branch-sensitive. Storing one global answer for a non-deterministic node is unsound; storing one answer per branch, task, or fingerprint preserves call-time choice while still removing redundant work.

3. Tabling, stateful refinement, and proof-assistant encodings

Memoization of non-deterministic choices is also the central mechanism behind tabling for recursive search. In Johnson’s continuation-passing approach, later reconstructed purely functionally in OCaml, a memo table for an input x stores both the answers found so far and the continuations waiting for new answers. The first call to p x creates an entry with empty answers and the current continuation; a recursive re-entry to p x does not recurse again, but registers a new continuation and replays the known answers. Whenever a new answer is discovered, it is added to the table and sent to all stored continuations. This is precisely how left recursion becomes a finite fixed-point computation instead of non-termination. The monadic interface exposes memo, memrec, and memrec2, while the internal stack ContT Dynamic (ListT Ref) implements the answer-and-continuation table in a purely functional setting (Abdallah, 2017).

Choice Trees provide a second, more denotational perspective. A ctree has constructors Ret, Vis, brS, and brD, where brS and brD are two forms of internal nondeterministic branching. The paper does not implement explicit memoization, but it identifies the exact hook where memoization belongs: a stateful refinement refine_state into stateT St (ctree E) can record which branch of each brS or brD node was taken and then reuse that decision. The authors prove that this stateful refinement is a correct implementation of the original nondeterministic specification, so memoization appears as a refinement that commits to one branch while preserving the ctree’s bisimulation-theoretic meaning (Chappe et al., 2022).

In Agda, two encodings of Curry-style non-determinism separate two memoization granularities. The first represents all possible outcomes by a datatype ND A with constructors Val and _??_; this naturally supports memoization of the full result set for a given input. The second represents a non-deterministic function as Choice -> A -> B, with abstract selectors choose, lchoice, and rchoice; this encodes every non-deterministic decision explicitly and is therefore better suited to memoizing individual runs or choice sequences. The paper presents the two approaches as “distinct and competitive,” with the intensional Choice encoding tracking every non-deterministic choice that the application could perform (Antoy et al., 2017).

Taken together, these approaches suggest two recurrent memoization targets. One may cache the set of all answers of a non-deterministic computation, or instead cache the resolution of each choice point. The first is extensional and tabling-oriented; the second is intensional and execution-oriented.

4. Distribution-wide and protocol-wide commitment

In process calculi, memoization of non-deterministic choices often becomes a restriction on how observers or protocol participants may branch. In Linear Quantum CCS, the initial saturated semantics allowed a context to contain its own non-deterministic choice and to select different branches on different configurations of the same distribution. The paper shows that this gives observers unphysical power: they can behave as if they knew hidden quantum values without measuring them. The refined semantics introduces indexed transitions \longsquiggly_\pi and a lifting condition requiring that every configuration in the support of a distribution take the same observer index \pi. Operationally, the observer’s non-deterministic choice is therefore global to the distribution and is reused across all quantum branches unless later classical information justifies divergence. The resulting constrained bisimilarity \sim_{cs} is coarser than the original saturated bisimilarity and, according to the paper, is the first semantics to both lift the indistinguishability of quantum states to distributions of processes and preserve the expressivity of non-deterministic choices based on classical information (Ceragioli et al., 2023).

A session-typed concurrent calculus provides an analogous notion of commitment at protocol scope. The eager semantics for non-deterministic choice uses ND-contexts and a commitment operation that erases all non-selected branches in the context of a synchronization. Once one branch of P \Box Q participates in communication, the surrounding ND-context collapses and the discarded branches are literally removed from the process term. This is not presented as memoization in the paper, but it functions as a once-and-for-all commitment mechanism: a resolved protocol choice cannot later be replayed differently. The key meta-results are type preservation and deadlock-freedom under this eager semantics, showing that linearly typed resources can survive branch commitment without duplication or loss (Heuvel et al., 2024).

A related but more language-theoretic viewpoint appears in automata. For an NFA on finite words, a history-deterministic resolver is a function that, given the current state and next input symbol, chooses one outgoing transition so that if some accepting run exists on a word, the resolver-induced run also accepts. In the stochastic generalization, a memoryless stochastic resolver assigns probabilities to transitions, and \lambda-resolvability requires every accepted word to be accepted with probability at least \lambda. The paper observes that 1-resolvability on finite words coincides with history-determinism, while smaller \lambda yield a strict quantitative hierarchy. This is another form of memoized nondeterminism: a fixed policy replaces extensional existential branching by a reusable choice rule indexed by local history (Paul et al., 14 Apr 2025).

Across these settings, commitment is the shared abstraction. Whether the index is a distribution label \pi, an ND-context collapse, or a resolver table over state-letter pairs, the semantic effect is to forbid later independent re-resolution of the same non-deterministic opportunity.

5. Memoized choice as proof object, certificate, or policy

In classical arithmetic with dependent types and control operators, memoization is elevated to a proof-theoretic principle. The calculi dPA^{\omega} and dLPA^{\omega} encode countable choice and dependent choice by building coinductive streams whose cells are evaluated lazily and shared through a store. A proof of \forall n\,\exists y\,P(n,y) can behave differently in different continuations because of control operators, so naively calling it twice at the same n may produce inconsistent witnesses. The stream construction H_\infty avoids this by storing each H\,n : \exists y^T.P(n,y) once; each cell is unfolded only on demand, then memoized and reused everywhere. The resulting realizers for AC_{\mathbb{N}} and DC are therefore classical but computationally stable, and the sequent-calculus presentation supports normalization and soundness proofs via Krivine-style realizability with stores (Miquey, 2019).

Promise Algebra makes the same idea explicit in algebraic form. A history-dependent Choice function is fixed for an entire computation; once fixed, it resolves every atomic non-deterministic transition in a way that is consistent with the current history. Equivalence classes of such functions are witnesses, promises, or certificates. Semantically, a Choice function behaves like a memo table keyed by history and module symbol: for a given history, the chosen successor is fixed and reused in all subterms of the same term. This turns global resolution of non-determinism into a first-class semantic object (Ternovska, 2023).

Dodona reinterprets non-deterministic choice as a memoizable state for search and learning. A choose expression does not immediately pick an alternative; evaluation returns either a value or a choicepoint consisting of a list of possible values and a continuation stack. The paper describes this choicepoint as a lossless encoding of all information necessary in principle to make an optimal decision. Although the implementation does not provide an explicit cache from choicepoints to oracle outputs, the universal policy is exactly a function from choicepoints to policy and value estimates, so it acts as amortized reuse of past decision structure across runs and tasks (Selsam et al., 2020).

These three lines of work show that memoization of non-deterministic choices is not confined to runtime engineering. It can be a proof object in arithmetic, a certificate in algebraic semantics, or a state abstraction for learned heuristic control.

6. Expressive power, decidability, and open semantic boundaries

Memoization policy affects expressive power. In cons-free higher-order implicit complexity, deterministic programs of data order K characterize EXP^{K}. When explicit nondeterministic choice is added, the situation changes sharply: for data order at least 1, cons-free programs with explicit nondeterminism characterize the entire class ELEMENTARY. The construction relies on interpreting higher-order values extensionally as relations rather than functions, so repeated application of the same higher-order object can explore different nondeterministic behaviors. The paper does not study memoization directly, but it explicitly ties the expressive jump to repeated higher-order nondeterministic behavior, which suggests that memoizing such choices would not be semantically innocuous (Kop et al., 2017).

A complementary open boundary appears in process algebra with probability distributions over data. The generalized sum \sum_{d:D} p(d) is a binding form, so within the continuation the chosen value is fixed. The unresolved issue is how infinitely many such choices should cohere globally in the presence of probability density functions. The “Real Hotel” example exhibits the problem: the natural semantics seems to require an unbounded product of density functions, but the paper states that it is unclear what such a product is, or even whether, or under which circumstances, it actually exists. In the paper’s own phrasing, how one globally “memoizes” or correlates infinitely many probabilistic and non-deterministic choices remains an open semantic design question (Groote, 2021).

Decidability results in automata strengthen this point. For general NFAs, deciding \lambda-memoryless stochastic resolvability is undecidable. For finitely ambiguous automata it is decidable, while positive resolvability is PSPACE-complete; for unambiguous NFAs, positive resolvability is NL-complete and \lambda-resolvability is in PTIME. The paper also proves that, for every rational \lambda \in (0,1), there exists a unary NFA that is \lambda-resolvable but not (\lambda+\varepsilon)-resolvable for any \varepsilon\>0. Thus even the ability to replace existential branching by a reusable stochastic resolver exhibits a strict hierarchy and hard decision frontiers (Paul et al., 14 Apr 2025).

A plausible implication is that memoization of non-deterministic choices should be viewed as a semantic resource with multiple axes: scope, correlation, and persistence. Scope asks whether one shares a choice across one call, one branch, one distribution, or an entire computation. Correlation asks whether different occurrences remain independent or must reuse a common decision. Persistence asks whether the commitment survives later control effects, protocol interactions, or learned policy reuse. Much of the modern literature can be read as the systematic study of these three axes.

Memoization of non-deterministic choices therefore occupies a central position between semantics and implementation. In some settings it enforces call-time choice, tabling, or proof stability; in others it constrains observers, commits protocols, or replaces existential branching by policies and certificates. Its central technical role is always the same: to determine when one non-deterministic source denotes one reusable decision and when it denotes many fresh computations.