Stevie: Automated Higher-Order Refactoring

• Stevie is an automated system that refactors first-order logic programs by inventing reusable higher-order abstractions (map, filter, fold) to compress program definitions while preserving semantics.
• It reformulates the refactoring task as a constraint optimization problem, systematically selecting abstractions and instantiations using CP-SAT to minimize program size.
• Empirical evaluations show significant improvements, including a 27% boost in predictive accuracy and a 47% reduction in learning time across varied program synthesis tasks.

Stevie is an automated system for higher-order refactoring of logic programs, designed to invent and introduce higher-order abstractions such as map, filter, and fold in order to compress first-order logic programs while preserving their semantics on specified target predicates. By reformulating the refactoring process as a constraint optimization problem (COP), Stevie enables systematic discovery of reusable higher-order patterns, leading to significant improvements in the efficiency and generality of inductive logic programming (ILP) workflows (Hocquette et al., 2023).

1. Higher-order Refactoring Problem

The higher-order refactoring problem addressed by Stevie is as follows: given a first-order logic program $P$—i.e., a set of definite Horn clauses—automatically invent higher-order relations that replace recurring sub-patterns in $P$ with compact abstractions of the form map, filter, or fold. The aim is maximal compression of $P$'s definitions $\delta(P)$, subject to semantic preservation on a set of target predicates $T$. The key formal components are as follows:

• Abstraction ($a$): For a definition $d \in \delta(P)$, an abstraction replaces a subset $\{p_1, ..., p_n\}$ of body predicates by higher-order variables $X_1, ..., X_n$, yielding a new (fresh) head predicate $h'(v_1, ..., v_k, X_1, ..., X_n)$ with the original semantics (Definition 1).
• Instantiation ($P$0): Instantiating an abstraction binds each higher-order variable $P$1 to a concrete predicate $P$2 from the original program (Definition 2).
• Solution ($P$3): Any $P$4 (where $P$5 and $P$6 denote the sets of abstractions and instantiations, respectively) is a valid solution if its least Herbrand model on $P$7, $P$8, coincides with $P$9 (Definition 4).
• Optimality: The optimal solution minimizes program size and a penalty on the number of higher-order variables (Definition 5), reflecting the heuristic that smaller programs generalize better in ILP.

2. System Pipeline and Architecture

Stevie's operational pipeline consists of two sequential stages:

1. Abstract Stage: Given input program $P$0 and hyperparameter $P$1 (bounding the subset size), Stevie enumerates every subset $P$2 (up to size $P$3) of non-recursive body predicates in each definition. For each $P$4, the system uses create_abs_inst(d, \psi) to construct a candidate abstraction $P$5 and its instantiation $P$6. Pruning ensures duplicate (isomorphic) abstractions—differing only by predicate renaming—are discarded, outputting sets $P$7 (abstractions) and $P$8 (instantiations).
2. Compress Stage (COP): The refactoring process is encoded as a constraint optimization problem (Sections 3.1–3.3), where binary variables indicate the selection of abstractions and instantiations. The objective function is:

$P$9

where $\delta(P)$0, $\delta(P)$1, and $\delta(P)$2 are binary decision variables; $\delta(P)$3 counts literals, and $\delta(P)$4 is the number of higher-order variables in $\delta(P)$5. The penalty coefficient $\delta(P)$6 is fixed to 1. The underlying solver (CP-SAT via OR-Tools) optimizes this objective subject to constraints that ensure each definition is either refactored or retained, and each abstraction is used only if instantiated.

3. Constraint Optimization Formulation

Stevie's formal COP (Constraint Optimization Problem) is specified as:

• Objective: Minimize $\delta(P)$7 subject to:
  • $\delta(P)$8
  • $\delta(P)$9
  • $T$0

This formulation guarantees (Proposition 1) that any COP solution yields a refactored program $T$1 with the same target semantics as $T$2, while the solver finds the $T$3 of minimum structural cost (Theorem). The explicit penalty on higher-order variables is empirically critical for learning accuracy and runtime—removal of this penalty degrades performance.

4. Higher-Order Abstraction Discovery

Stevie automatically discovers classic higher-order programming patterns:

• map: Templates allow parametric element-wise operations over lists via higher-order predicates $T$4, instantiated as operations such as uppercase or increment.
• filter: Two-predicate abstractions enabling selective processing based on predicates $T$5 and $T$6 over list elements.
• fold: Abstractions carrying an accumulator, capable of expressing reductions such as sum or max.

Template detection is achieved by exhaustive enumeration over predicate subsets of bounded size, followed by instantiation mapping and template pruning. Scalability is managed by pruning isomorphic abstractions and penalizing the number of higher-order variables to avoid uninformative generality.

5. Empirical Evaluation

Stevie is evaluated on 176 list-manipulation program synthesis tasks (25% held out), following this procedure: first-order solutions are learned (with Hopper) on a subset of tasks; abstractions are then extracted via Stevie and supplied as background knowledge (BK) to Hopper when learning held-out tasks. Against baselines of (i) Hopper with no refactoring and (ii) Hopper with standard SWI-Prolog higher-order predicates (maplist, foldl, etc.), Stevie achieves:

• 27% absolute increase in predictive accuracy (e.g., from ~71% to ~90%)
• 47% reduction in learning time per task (e.g., from ~412 s to ~218 s with $T$7)
• Reduction in program size (from ~8 literals to ~4)
• Penalizing higher-order variables is essential for performance; omitting this penalty reduces both accuracy and speed.

Stevie can optimally refactor logic programs up to approximately 460 literals in around 16 minutes. For larger programs, optimal certification incurs high computational cost, though near-optimal solutions may be found rapidly.

6. Abstraction Transferability Across Domains

Abstractions discovered by Stevie exhibit significant cross-domain transfer. Using templates learned from program synthesis, Hopper was evaluated on 35 tasks spanning seven domains—chess tactics, ASCII art, string transformation, robot strategies, list and tree manipulation, arithmetic. Transferred abstractions never impaired predictive accuracy and resulted in improvements in 5 out of 7 domains (e.g., from 50% to 100% on line-drawing and string-letter-filter tasks, as well as chess map-filter). For instance, a learned ho_filter abstraction was directly reused for string manipulation and chess tasks via an invented predicate.

7. Limitations and Future Directions

Stevie's contributions are threefold:

1. Formalization of the higher-order refactoring problem for logic programs.
2. Development of a two-stage system for abstraction discovery and optimal selection via constraint optimization.
3. Empirical demonstration that automatically discovered higher-order templates can yield substantial ILP performance improvements and generalize across disparate domains.

Present limitations include the scalability of the solution process for very large programs, reliance on a compression-oriented cost function (with higher-order variable penalty) as a heuristic for abstraction "usefulness," and a manual choice of the enumeration bound $T$8. Prospective directions include development of cost functions guided by learning or predictive accuracy, enhanced COP encodings and solving heuristics, parallelization, and extension to richer forms of higher-order abstraction (Hocquette et al., 2023).