Herb.jl: Modular Synthesis Library

• Herb.jl is a modular program synthesis library in Julia that unifies diverse synthesis approaches by abstracting common components such as grammars, constraints, and search strategies.
• It enables rapid prototyping and comparative evaluation by allowing researchers to easily interchange and optimize individual modules without reengineering the entire workflow.
• Its innovative uniform tree structure efficiently groups similar candidate programs, reducing memory overhead and streamlining constraint propagation and search performance.

Herb.jl is a unifying program synthesis library written in Julia that addresses fragmentation in program synthesis research by facilitating modular, extensible design of synthesis algorithms. Unlike prior specialized and non-extensible synthesizers, Herb.jl enables rapid prototyping, reimplementation, and benchmarking of diverse synthesis approaches. It achieves unification by abstracting common components—such as grammars, specifications, constraints, search strategies, and program representations—into interoperable modules, creating an efficient infrastructure for both top-down, bottom-up, stochastic, and genetic program synthesis paradigms (Hinnerichs et al., 10 Oct 2025).

1. Architectural Overview

Herb.jl is structured as a collection of tightly coupled yet independently replaceable submodules. The design centers on several core abstractions:

• HerbSpecification: Encapsulates the synthesis problem, typically via input–output examples.
• HerbGrammar: Defines the set of derivation rules describing the target language (e.g., arithmetic expressions, string transformations).
• HerbInterpret: Transforms abstract syntax trees (ASTs) into executable Julia expressions, enabling semantic validation of candidate programs.
• HerbConstraints: Manages the specification and propagation of constraints, facilitating pruning of the search space and breaking symmetry (e.g., in commutative operations).
• HerbSearch: Implements multiple search strategies; candidates are enqueued using configurable priority functions and iteration mechanisms.

A distinctive element is the “uniform tree” data structure, wherein programs sharing a fixed “shape” (node configuration) are grouped. Nodes may remain undecided (“holes”) with domains specified by grammar, reducing memory requirements and simplifying constraint propagation.

2. Modularization and Extensibility

Each component of Herb.jl is designed for complete modularity, allowing researchers to substitute or augment individual modules without reengineering the entire synthesis workflow. For example:

• Exchanging search strategies (e.g., depth-first, breadth-first, probabilistic enumeration, genetic search) involves minimal changes restricted to the search iterator and its priority/fitness functions.
• Constraint propagation mechanisms can be independently tuned and optimized.
• Syntax–semantics separation is maintained; grammars generate candidate ASTs which are then interpreted in downstream modules for specification satisfaction assessment.

Uniform trees, introduced as a key technical innovation, encode groups of candidate programs differing only in terminal/operator choice while sharing overall structure, thus supporting efficient enumeration and constraint checking across large program spaces.

3. Canonical Use Cases

Three paradigmatic use cases demonstrate the utility of Herb.jl:

a) Synthesizing Programs from Simple Grammars

A simple integer arithmetic domain can be defined with a context-free grammar and input–output pairs (specification):

problem = Problem([... IOExamples ...]) grammar = @cfgrammar begin Int = 1 | 2 | x Int = Int + Int Int = Int * Int end iterator = BFSIterator(grammar, :Int, max_depth=5) solution, flag = synth(problem, iterator)

Here, BFSIterator instantiates a breadth-first search over derivations, returning ASTs converted to Julia expressions for semantic evaluation.

b) Reimplementing Existing Synthesis Algorithms

The Probe algorithm, which relies on a probabilistic context-free grammar (PCFG), is implemented using only a handful of lines. Its search is guided by likelihood computation, prioritizing enumeration of candidate programs according to their probability under the grammar (most-likely-first):

@programiterator MLFSIterator() <: TopDownIterator function priority_function(::MLFSIterator, grammar, current_program, parent_value) return -max_rulenode_log_probability(current_program, grammar) end

The prioritized search is iterated as follows (pseudocode):

function probe(grammar, starting_sym, problem; probe_cycles=3) for cycle in 1:probe_cycles iterator = MLFSIterator(grammar, starting_sym, …) promising_programs, result_flag = get_promising_programs_with_fitness(iterator, problem) if result_flag == optimal_program return solution end modify_grammar_probe!(promising_programs, grammar) end return nothing end

Probe cycles through search iterations, updating grammar rule probabilities to refine future enumeration.

c) Benchmarking and Comparative Evaluation

Standardized benchmarks, including SyGuS string transformation challenges, are loaded via HerbBenchmarks.jl. Synthesizers are evaluated in scripted loops, collecting timing and performance metrics with Julia’s facilities (e.g., @time macro) for comparative analysis across algorithmic variants.

4. Technical Foundations

Some salient technical details integral to Herb.jl include:

• Uniform Trees: Nodes encode sets of operator applications consistent with a fixed shape, leveraging hole filling and shared domains for efficient memory management and constraint checking (see Figure 1 in the referenced work).
• Separation of Syntax and Semantics: Grammar-based AST generation is decoupled from semantic interpretation, supporting flexible language definitions and evaluation procedures.
• Priority Functions (Probe Algorithm Example): The priority for search enumeration is defined as:

$$\text{priority}(\text{program}) = -\sum_{\text{rule} \in \text{program}} \log\_probability(\text{rule})$$

This is realized by recursively accumulating log-probabilities through multiple dispatch in Julia.

• Search Strategies: Top-down, bottom-up, stochastic, and genetic enumeration are interchangeable through iterator abstraction. Custom fitness/priority functions allow adaptation to diverse synthesis paradigms.

5. Comparison to Existing Synthesis Libraries

Herb.jl distinguishes itself in several dimensions:

Dimension	Herb.jl Approach	Existing Tools
Flexibility	Modularized grammars, specifications, constraints, search strategies, and interpretation	Often monolithic or non-extensible
Ease of Use	Concise Julia API leveraging metaprogramming and multiple dispatch	Specialized interfaces, extensive boilerplate
Performance	Memory-efficient uniform tree representation; independently optimizable modules	Memory consumption scales per candidate program
Reusability	Encourages remixing and sharing of modules across paradigms	Typically domain- or problem-specific

This modularity enables adaptation and benchmarking of novel and legacy approaches, mitigating redundant engineering effort and streamlining empirical comparison of synthesis strategies.

6. Prospective Development and Research Directions

Several avenues for future advancement are outlined:

• Expansion of synthesis paradigms: Incorporation of neural-symbolic modules and learning-based synthesis strategies.
• Benchmark Diversification: Extending HerbBenchmarks.jl to cover emerging domains in program synthesis research.
• Search Optimization: Development of advanced heuristics and adaptive (possibly online) grammar probability learning.
• Julia Platform Enhancement: Ongoing exploitation of Julia's metaprogramming and multiple dispatch for reduction of code duplication and improvement of clarity and computational speed.

These directions aim to further generalize Herb.jl's capacity for unifying disparate synthesis methodologies and fostering robust comparative evaluation.

7. Context and Significance

Herb.jl represents a practical step toward standardized, extensible infrastructures in program synthesis research. By enabling rapid prototyping, seamless interchange of synthesis submodules, and comparative benchmarking, the library increases reproducibility and efficiency in the development of new algorithms. A plausible implication is that Herb.jl’s approach will facilitate both academic and industrial exploration of synthesis techniques, advancing the state-of-the-art in automated program generation through modular, reusable, and performance-conscious design (Hinnerichs et al., 10 Oct 2025).