Generator Module for Exact Synthetic Data Sampling

• Generator Module (Gen) is a core component in TreeGen that builds a probability tree to model the exact joint distribution of categorical data.
• It implements a Monte Carlo tree traversal algorithm to generate synthetic records that preserve observed conditional and marginal frequencies.
• The module offers fast, parallelizable generation with full interpretability and exact frequency matching, while scalability may be challenged in high-dimensional settings.

The Generator Module (“Gen”) is the central component of TreeGen, a Python package designed for exact, interpretable Monte Carlo sampling over the empirical joint distribution of a categorical data frame. Gen operates by traversing a compact probability-tree structure derived directly from the observed frequencies of unique rows in the data set, facilitating precise generation of synthetic records, data augmentation, compression, and feature extraction that faithfully respect all empirical conditional and marginal distributions.

1. Probability Tree Data Structure

Let a data frame $D$ have $m$ ordered columns $C_1, \dots, C_m$ and $N$ rows. The probability tree $T$ is a rooted tree of depth $m$, with each root-to-leaf path $(v_1, \dots, v_m)$ uniquely corresponding to a possible record $(C_1=v_1, \dots, C_m=v_m)$ with frequency exactly matching its empirical count in $D$.

Each node at level $k$ encodes:

• columnName = C_{k+1}
• A list of dataNode objects, each with:
  • `value = v$m$0C_{k+1}$m$1[v_1,\dots, v_k]$)</li> <li>`probability = p(v \mid \text{path}_k)$m$2\text{path}_k = (v_1, ..., v_k)$m$3 child Node for level $m$4 (built from rows of $m$5 satisfying $m$6)

Branch probabilities are computed as

$m$7

where $m$8 is the number of rows in $m$9 matching the prefix and $C_1, \dots, C_m$0.

Because the tree stores conditional probabilities at each node, the full joint for a path $C_1, \dots, C_m$1 is

$C_1, \dots, C_m$2

which guarantees that all empirical joint, marginal, and conditional frequencies are exactly recoverable by the tree’s structure.

2. Monte Carlo Tree Traversal Algorithm

The Gen module implements sampling by random walks through the probability tree:

• Begin at the root node.
• At each level $C_1, \dots, C_m$3, randomly select a child according to the conditional probabilities stored in the current node.
• Continue recursively until a leaf is reached, thus generating a complete synthetic record.

Pseudocode for generating a single record: $N$7 Each call selects one value per column, and because choices are sequentially made based on empirical empirical conditionals, generated samples preserve the joint-dependence structure in expectation.</p> <p>Gen manages a private pseudo-random number generator (<code>random.Random</code> or NumPy RNG), enabling bitwise-reproducible runs via explicit seed control.</p> <p>Generated statistics (marginals, conditionals) converge to those of the original data via the law of large numbers.</p> <h2 class='paper-heading' id='implementation-and-complexity'>3. Implementation and Complexity</h2> <p>The TreeGen implementation utilizes the following class architecture:</p> <div class='overflow-x-auto max-w-full my-4'><table class='table border-collapse w-full' style='table-layout: fixed'><thead><tr> <th>Class</th> <th>Attributes/Methods</th> <th>Functionality</th> </tr> </thead><tbody><tr> <td>Node</td> <td>columnName (str), data (list of dataNode)</td> <td>Tree node at each column</td> </tr> <tr> <td>dataNode</td> <td>value (category), probability (float), nextNode (Node or None)</td> <td>Branch from a node</td> </tr> <tr> <td>ProbabilityTree</td> <td><code>__init__</code>, <code>_build_subtree</code>, <code>getTree</code>, <code>getColumns</code></td> <td>Tree construction from a DataFrame</td> </tr> <tr> <td>Generator</td> <td><code>__init__</code>, <code>setSeed</code>, <code>getRecord</code>, <code>getRecords</code></td> <td>Monte Carlo generation of synthetic records</td> </tr> </tbody></table></div> <p>Complexity:</p> <ul> <li>Tree construction: $C_1, \dots, C_m$4 (per-level pass with sorting/hashing)

Space: $C_1, \dots, C_m$5 ($C_1, \dots, C_m$6 = maximum cardinality per column), often far less in practice (sparse tables)

Generation per record: $C_1, \dots, C_m$7; typically $C_1, \dots, C_m$8 if $C_1, \dots, C_m$9 is small

Output: Generating $N$0 samples costs $N$1 time, $N$2 output space

4. Usage Scenarios and Workflows

TreeGen supports a concise data-augmentation and synthetic data generation pipeline:

$N$8

Principal applications include:

• Data multiplicity increase for downstream ML or statistical modeling
• Bayesian compression: efficient data storage via tree-based representation
• Visualization of categorical interaction structure
• Hierarchical, nonparametric modeling of relationships among columns
• Feature extraction (e.g., most probable record, via tree.getMaxRecord())

5. Advantages, Trade-offs, and Limitations

Comparative advantages:

• Multi-way splits: Unlike binary decision trees, every node supports arbitrary degree (number of children), facilitating faithful modeling of categorical variables with high arity.
• Exact empirical matching: No smoothing, binning, or parametric assumptions; all conditional probabilities reflect observed frequencies.
• Fast, trivially parallelizable generation: Each synthetic record is independent (given PRNG seed).
• Full interpretability: All probabilities exposed in tree structure, enabling direct inspection of dependencies.

Known limitations:

• Scalability: Combinatorial blowup as both $N$3 and per-column cardinalities increase; memory and construction costs may be prohibitive for very high-dimensional or high-cardinality data.
• Memory usage: Each unique observed prefix (partial record) yields a node; worst-case $N$4 nodes.
• Construction overhead: For very large datasets ($N$5) or many columns ($N$6), initial tree-building can become a bottleneck.
• No generalization beyond support: Unseen value combinations in training have zero probability in generated data. Smoothing, hybrid modeling, or back-off strategies are required if extrapolation beyond observed support is desired.

6. Significance and Context

The Gen module in TreeGen addresses the problem of preserving empirical multivariate categorical dependencies in synthetic data generation, without the opacity or ad-hoc constraints of typical parametric or tree-split approaches. Its strict nonparametricity and full transparency make it particularly suitable for statistical data compression, exact simulation studies, and tasks requiring preservation of conditional frequency structure. Unlike decision trees, which generally enforce binary splits and do not encode empirical frequencies, Gen’s probability tree achieves maximum fidelity at the cost of possible combinatorial expansion in state-space. This positions TreeGen’s Gen as a robust tool for small to moderate-dimensional categorical datasets where interpretability and exact reproduction are essential (Niemczynowicz et al., 2020).