Circuit Tokenizer (CKT) Techniques

• Circuit Tokenizer (CKT) is a specialized process that converts structured circuit representations into discrete tokens for generative modeling, synthesis, and design automation.
• It encompasses distinct methodologies including latent node-level quantization in logic synthesis, symbolic tokenization in AIG traversal, and motif-based compression in analog topology generation.
• These tokenization techniques determine circuit representation and structural invariants, influencing search space reduction, synthesis efficiency, and design accuracy across EDA flows.

Searching arXiv for the cited papers and closely related terminology to ground the article in the latest literature. arXiv search query: (Wu et al., 18 Feb 2025) Circuit Tokenizer (CKT) denotes a class of circuit-specific tokenization mechanisms that convert structured circuit objects into discrete symbols suitable for generative modeling, synthesis, or language-model training. In recent arXiv literature, the term has three closely related but non-identical uses. In truth-table-guided logic synthesis, the tokenizer is the vector-quantized graph module called CircuitVQ, which is explicitly noted as synonymous with “Circuit Tokenizer (CKT)” elsewhere (Wu et al., 18 Feb 2025). In analog topology generation, CKT is a graph-aware tokenizer that mines frequent subcircuits from SPICE netlists and turns them into macro tokens (Islam et al., 7 May 2026). A third line of work, "Circuit Transformer," does not name a standalone CKT, but specifies a concrete token vocabulary, positional scheme, and constrained decoder that jointly play an analogous role for And-Inverter Graph generation (Li et al., 2024). Across these works, tokenization is not a peripheral preprocessing step: it determines the representation target, the structural invariants preserved by the sequence or code indices, and the extent to which downstream generation can enforce equivalence, compress topology, or guide search.

1. Terminological scope and representational regimes

Recent work uses circuit tokenization in three representational regimes: latent node-level discretization of logic DAGs, explicit traversal-token serialization of AIGs, and motif-based macro-tokenization of analog netlist hypergraphs (Wu et al., 18 Feb 2025, Li et al., 2024, Islam et al., 7 May 2026). The differences are substantive. They affect whether tokens correspond to learned codes or symbolic primitives, whether correctness is recovered by post-hoc optimization or guaranteed during decoding, and whether the tokenizer is intended for exact Boolean synthesis or topology-level analog generation.

Work	Representation target	Tokenization unit
CircuitVQ / CKT	DAG with PI, PO, and NAND nodes	One learned VQ token per node
Circuit Transformer	AIG serialized by depth-first traversal	Explicit tokens for PI/complements, AND, NAND, EOS, PAD
CircuitFormer CKT	SPICE netlist as typed hypergraph over nets	Primitive device tokens and learned macro-subcircuit tokens

This diversity makes “CKT” a family resemblance term rather than a single standardized module. In logic-synthesis settings, the tokenizer is tightly coupled to exact truth-table equivalence and gate-library assumptions. In analog settings, it is coupled instead to connectivity preservation, permutation invariance, and compression of recurring motifs. A plausible implication is that circuit tokenization must be defined relative to both the circuit formalism and the downstream objective, rather than imported unchanged from generic sequence modeling.

2. CircuitVQ as a node-level tokenizer for truth-table-guided logic synthesis

In "Architect of the Bits World: Masked Autoregressive Modeling for Circuit Generation Guided by Truth Table," the circuit tokenizer is called CircuitVQ and is built on a Circuit AutoEncoder (Wu et al., 18 Feb 2025). The circuit is modeled as a directed acyclic graph $G=(V,A)$ with binary adjacency matrix $A \in \{0,1\}^{N \times N}$, where nodes are restricted to three types: primary inputs, primary outputs, and NAND gates. This NAND-only formulation is motivated by logical completeness. Structural constraints used later in DAG Search and DAS initialization include indegree $0$ for PIs, outdegree $0$ for POs, prohibition of self-loops, and iterative cycle removal by deleting the lowest-probability edge in each detected cycle.

The encoder $g_E$ maps $G=(V,A)$ to continuous node embeddings $Z \in \mathbb{R}^{N \times d}$ using Graphormer:

$Z = g_E(V,A).$

The decoder $g_D$ is a deliberately weak Transformer-based module that reconstructs only the adjacency matrix, because node types are known from the truth table. If $\tilde{X}=g_D(\cdot)$, adjacency is reconstructed as

$A \in \{0,1\}^{N \times N}$0

yielding a probabilistic $A \in \{0,1\}^{N \times N}$1. The reconstruction objective is binary cross-entropy over edges:

$A \in \{0,1\}^{N \times N}$2

CircuitVQ inserts a vector quantizer $A \in \{0,1\}^{N \times N}$3 with codebook $A \in \{0,1\}^{N \times N}$4 and uses one token per node. The full circuit is therefore represented by a token sequence

$A \in \{0,1\}^{N \times N}$5

Code selection uses $A \in \{0,1\}^{N \times N}$6 normalization, equivalently cosine similarity:

$A \in \{0,1\}^{N \times N}$7

The paper uses $A \in \{0,1\}^{N \times N}$8 codes and codebook embedding dimension $A \in \{0,1\}^{N \times N}$9, followed by remapping to the decoder dimension. The VQ objective combines reconstruction with codebook and commitment terms in standard VQ-VAE style, and because quantization is non-differentiable, gradients are copied from decoder input to encoder output with a straight-through estimator. To mitigate codebook collapse, the design uses $0$0-normalized nearest-neighbor lookup in reduced dimension and Gumbel-Softmax during training.

The architectural choices are highly specific. The encoder is a Graphormer with $0$1 encoder layers, hidden dimension $0$2, FFN dimension $0$3, and $0$4 attention heads; the decoder has $0$5 Transformer decoder layer. Training uses a dataset of approximately $0$6 circuits with average $0$7 gates per circuit, constructed from ISCAS’85, IWLS’05, and EPFL benchmarks by multi-output subcircuit extraction in AIG, with maximum input size $0$8 and at most $0$9 distinct topologies per truth table. Augmentation includes random node order shuffles and insertion of idle NAND gates with ratio randomly $0$0–$0$1. Optimization uses AdamW with learning rate $0$2, weight decay $0$3, linear schedule, batch size $0$4, and $0$5 epochs on $0$6A100 (80G) GPUs. In this formulation, the tokenizer is a learned latent structural code rather than a symbolic grammar.

3. Masked autoregression, detokenization, and DAS guidance

CircuitVQ is designed to support CircuitAR, a masked autoregressive model conditioned on truth tables (Wu et al., 18 Feb 2025). CircuitAR is a bidirectional Transformer variant described as Llama-style; each layer has self-attention, cross-attention, and FFN blocks, with the truth table $0$7 fed as key/value to the cross-attention block. The training target is the CircuitVQ token sequence $0$8.

Masked autoregressive training replaces a subset of positions by a special mask token $0$9 using the cosine schedule

$g_E$0

with mask size $g_E$1. The loss is

$g_E$2

where $g_E$3 is the masked sequence and $g_E$4 the masked node set. Decoding is parallel masked decoding: starting from $g_E$5, the model predicts all tokens in parallel, keeps the most confident subset according to the cosine schedule, and re-masks the rest for the next iteration until a full sequence is produced.

Detokenization converts the decoded tokens back into structure through the CircuitVQ decoder, producing $g_E$6 and then the probabilistic adjacency

$g_E$7

Algorithmically, this probabilistic adjacency is not used as a final circuit. It is first passed through DAG Search: threshold edges greater than $g_E$8, enforce PI/PO/self-loop constraints, detect cycles by DFS, and iteratively remove the lowest-probability edge within each cycle until a valid DAG $g_E$9 remains. The method also forms

$G=(V,A)$0

the elementwise product aligning probabilities to the DAG edges. Levelization yields logic levels $G=(V,A)$1, and DAS is initialized by normalizing connection probabilities to previous-layer nodes and setting Softmax logits to

$G=(V,A)$2

with $G=(V,A)$3 the number of previous-layer candidate inputs. If DAS stalls in a local optimum, the least-confident nodes are reinitialized by the same formula.

The reported effect is that the tokenizer becomes a structural prior for differentiable search rather than a complete synthesis engine. CircuitAR-2B achieves an average $G=(V,A)$4 reduction in DAS steps over CircuitNN, compared with $G=(V,A)$5 for T-Net, while maintaining $G=(V,A)$6 functional accuracy across IWLS benchmarks. Example step reductions include $G=(V,A)$7 for ex46 and $G=(V,A)$8 for ex01. It reduces the search space by $G=(V,A)$9 on average and used NAND gates by $Z \in \mathbb{R}^{N \times d}$0 compared to CircuitNN; for ex16, NAND usage drops by $Z \in \mathbb{R}^{N \times d}$1, and for ex92, by $Z \in \mathbb{R}^{N \times d}$2. The Bits Distance metric improves from $Z \in \mathbb{R}^{N \times d}$3 at $Z \in \mathbb{R}^{N \times d}$4 parameters to $Z \in \mathbb{R}^{N \times d}$5 at $Z \in \mathbb{R}^{N \times d}$6, and data scaling improves the $Z \in \mathbb{R}^{N \times d}$7 model from $Z \in \mathbb{R}^{N \times d}$8 to $Z \in \mathbb{R}^{N \times d}$9, an $Z = g_E(V,A).$0 gain. An initialization ablation reduces average DAS steps from $Z = g_E(V,A).$1 without initialization to $Z = g_E(V,A).$2 with initialization while preserving $Z = g_E(V,A).$3 functional accuracy. The paper also reports an emergent capability near $Z = g_E(V,A).$4 parameters: deeper circuits with fewer NAND gates, interpreted as improved topological optimization.

4. Equivalence-preserving tokenization in Circuit Transformer

"Circuit Transformer: A Transformer That Preserves Logical Equivalence" does not define a separately named “Circuit Tokenizer (CKT),” but it specifies a concrete tokenization scheme and a constrained decoder that together serve the same role in AIG generation (Li et al., 2024). The representation target is an And-Inverter Graph. Each primary output is treated as the root of a binary tree of AND/NAND nodes with leaves at primary inputs or their complements, and the circuit is serialized by a depth-first, memory-less traversal.

For an $Z = g_E(V,A).$5-input circuit, the vocabulary size is $Z = g_E(V,A).$6 and contains EOS, PAD, the PI tokens $Z = g_E(V,A).$7 and their complements $Z = g_E(V,A).$8, and the gate tokens $Z = g_E(V,A).$9 and $g_D$0. There are no separate inverter or wire tokens. Structure is implicit in the pre-order traversal and maintained by a path stack: gate tokens push a “need two children” frame, PI tokens fill child slots, and completed gates pop from the stack. Multiple POs are serialized consecutively. Positional encoding is tree-structured rather than linear: the current path from the PO root to the expansion site is encoded by a stack of two-bit one-hot inputs, “10” for first-input branch and “01” for second-input branch, with a unique one-hot code prepended for each PO root.

The critical addition is the cutoff mechanism based on tri-valued partial evaluation over a constraint set of input-output pairs. Logic values are in $g_D$1, where $g_D$2 denotes “not determined yet.” For a candidate token $g_D$3 at time $g_D$4, let $g_D$5 be the value of the active PO root under constraint $g_D$6 after tentatively inserting $g_D$7. The token is blocked iff

$g_D$8

This yields three invariants: a depth-first invariant, an equivalence invariant, and a stack invariant. When the constraint set enumerates the full truth table, the decoder guarantees

$g_D$9

The guarantee comes from the masking rule, not merely from the token vocabulary.

The model is an encoder-decoder Transformer with $\tilde{X}=g_D(\cdot)$0 layers each, embedding width $\tilde{X}=g_D(\cdot)$1, feedforward size $\tilde{X}=g_D(\cdot)$2, and approximately $\tilde{X}=g_D(\cdot)$3 parameters. It is trained on $\tilde{X}=g_D(\cdot)$4 randomly generated $\tilde{X}=g_D(\cdot)$5-input, $\tilde{X}=g_D(\cdot)$6-output AIGs, with teacher forcing on target synthesized trajectories produced by resyn2 in ABC, batch size $\tilde{X}=g_D(\cdot)$7, learning rate $\tilde{X}=g_D(\cdot)$8, and $\tilde{X}=g_D(\cdot)$9 training batches. Inference uses the constrained decoder, optionally augmented by Monte-Carlo Tree Search under an MDP formulation where gate insertions receive reward $A \in \{0,1\}^{N \times N}$00 and PI insertions receive reward $A \in \{0,1\}^{N \times N}$01, with additional reward for gate merging due to fanout reuse.

The evaluation demonstrates that tokenization and constrained decoding can preserve exact correctness without a separate post-optimization stage. On $A \in \{0,1\}^{N \times N}$02 held-out $A \in \{0,1\}^{N \times N}$03-input, $A \in \{0,1\}^{N \times N}$04-output circuits with sequence-length cap $A \in \{0,1\}^{N \times N}$05, resyn2 has average AND count $A \in \{0,1\}^{N \times N}$06, Circuit Transformer alone $A \in \{0,1\}^{N \times N}$07, and Circuit Transformer with MCTS at $A \in \{0,1\}^{N \times N}$08 steps and $A \in \{0,1\}^{N \times N}$09 rollouts achieves average $A \in \{0,1\}^{N \times N}$10. The equivalence violation rate is reported as zero under the full-truth-table cutoff mask. This separates the work from CircuitVQ-based synthesis: the latter tokenizes structure to guide DAS, whereas Circuit Transformer enforces exact equivalence during generation itself.

5. Graph-aware CKT for analog netlist topology in CircuitFormer

In "CircuitFormer: A Circuit LLM for Analog Topology Design from Natural Language Prompt," CKT is explicitly introduced as a “novel circuit graph tokenizer” for analog netlists (Islam et al., 7 May 2026). The motivation is that general-purpose tokenizers such as BPE or SentencePiece are structurally blind: they operate on linear text, depend on arbitrary line ordering and whitespace, and do not encode which devices share nets or which motifs recur. Earlier circuit-specific serializations are also criticized for vocabulary growth proportional to the maximum number of components $A \in \{0,1\}^{N \times N}$11 in the dataset.

CKT instead treats a SPICE netlist as a hypergraph $A \in \{0,1\}^{N \times N}$12 over nets. Ground is canonicalized to node $A \in \{0,1\}^{N \times N}$13. Each edge is a typed hyperedge $A \in \{0,1\}^{N \times N}$14, where $A \in \{0,1\}^{N \times N}$15 is the device type and $A \in \{0,1\}^{N \times N}$16 is the ordered port tuple. For topology-only compression metrics, device parameters and model values are stripped. Preprocessing removes comments and directives, merges continuation lines, remaps net identifiers to contiguous integers, canonicalizes ground, and anonymizes instance names so that, for example, $A \in \{0,1\}^{N \times N}$17, $A \in \{0,1\}^{N \times N}$18, and $A \in \{0,1\}^{N \times N}$19 become $A \in \{0,1\}^{N \times N}$20. For F, H, and K elements, dummy nodes are inserted so that controlling branches referenced by instance name can be encoded topologically and later reconstructed losslessly.

The core mining algorithm adapts grammar-based graph compression via gRePair. All training netlists are aggregated by disjoint union:

$A \in \{0,1\}^{N \times N}$21

CKT enumerates adjacent hyperedge pairs, or digrams, that share at least one net and distinguishes them using shared-node masks to preserve series/parallel structure and port-order semantics. Selection is driven by Minimum Description Length gain. For a digram $A \in \{0,1\}^{N \times N}$22, with span $A \in \{0,1\}^{N \times N}$23 primitive devices and frequency $A \in \{0,1\}^{N \times N}$24 non-overlapping occurrences,

$A \in \{0,1\}^{N \times N}$25

A merge is performed only if $A \in \{0,1\}^{N \times N}$26, subject to support threshold $A \in \{0,1\}^{N \times N}$27 and maximum macro arity $A \in \{0,1\}^{N \times N}$28. Each accepted digram becomes a non-terminal macro token with an ordered tuple of external nets. Token sequences are then emitted as flat edge lists with concrete port nets, such as $A \in \{0,1\}^{N \times N}$29 or $A \in \{0,1\}^{N \times N}$30; detokenization recursively expands macros and reconstructs branches, references, names, and values.

A central formal claim is the vocabulary-complexity separation from instance-specific baselines. For CKT,

$A \in \{0,1\}^{N \times N}$31

so for fixed $A \in \{0,1\}^{N \times N}$32 it is $A \in \{0,1\}^{N \times N}$33 with respect to circuit size. By contrast, instance-specific schemes satisfy

$A \in \{0,1\}^{N \times N}$34

hence $A \in \{0,1\}^{N \times N}$35. The paper summarizes the practical tokenization pipeline as roughly $A \in \{0,1\}^{N \times N}$36 overall, linear in graph size per iteration and capped by a constant number of learned rules.

Empirically, the chosen operating point is a fixed vocabulary of size $A \in \{0,1\}^{N \times N}$37. At this point, CKT achieves compression ratio $A \in \{0,1\}^{N \times N}$38 versus GPT-4 tiktoken BPE and sequence length reduction $A \in \{0,1\}^{N \times N}$39. With base primitives only, $A \in \{0,1\}^{N \times N}$40; with maximum mined vocabulary of approximately $A \in \{0,1\}^{N \times N}$41 tokens, $A \in \{0,1\}^{N \times N}$42, but later generation quality declines due to token sparsity. The overall dataset contains $A \in \{0,1\}^{N \times N}$43 netlist–natural-language pairs spanning $A \in \{0,1\}^{N \times N}$44 major SPICE element types, and all netlists are syntactically verified in Ngspice. CircuitFormer itself is a $A \in \{0,1\}^{N \times N}$45 parameter encoder-decoder Transformer with an MPNet/sBERT-style semantic encoder and a GPT-2 decoder with cross-attention, trained in three stages on a single NVIDIA H100 96GB. On CircuitBench-100 it achieves $A \in \{0,1\}^{N \times N}$46 validity and $A \in \{0,1\}^{N \times N}$47 functional success, outperforming open-source LLMs by $A \in \{0,1\}^{N \times N}$48 and $A \in \{0,1\}^{N \times N}$49, respectively, while using approximately $A \in \{0,1\}^{N \times N}$50 fewer parameters. Replacing CKT with a $A \in \{0,1\}^{N \times N}$51-vocab BPE lowers success from $A \in \{0,1\}^{N \times N}$52 to approximately $A \in \{0,1\}^{N \times N}$53–$A \in \{0,1\}^{N \times N}$54, and disabling line-shuffle augmentation lowers success to approximately $A \in \{0,1\}^{N \times N}$55 with validity approximately $A \in \{0,1\}^{N \times N}$56.

6. Comparative significance, misconceptions, and limitations

A common misconception is that “Circuit Tokenizer (CKT)” refers to a single canonical component. The literature instead presents at least three distinct design patterns. CircuitVQ is a learned node-level latent discretizer for NAND-DAGs, optimized to provide structural priors to a downstream differentiable search pipeline; the Circuit Transformer scheme is a symbolic AIG tokenizer whose correctness depends on a tri-valued cutoff mask; and CircuitFormer’s CKT is a graph-compression tokenizer for analog hypergraphs that emphasizes connectivity and motif reuse rather than exact Boolean equivalence (Wu et al., 18 Feb 2025, Li et al., 2024, Islam et al., 7 May 2026). These are not interchangeable modules.

A second misconception is that tokenization by itself guarantees circuit correctness. The papers distinguish three different correctness notions. In the CircuitAR pipeline, the generated token sequence yields an initial structure that guides DAS to exact solutions; exact equivalence is therefore achieved after differentiable optimization against the truth table, not by the tokenizer alone (Wu et al., 18 Feb 2025). In Circuit Transformer, strict equivalence is preserved during decoding because invalid tokens are masked by the cutoff rule under the full truth table (Li et al., 2024). In CircuitFormer, the headline metrics are $A \in \{0,1\}^{N \times N}$57 syntactic correctness and $A \in \{0,1\}^{N \times N}$58 functional success on analog tasks, which is a different guarantee from exact logical equivalence (Islam et al., 7 May 2026).

The limitations are correspondingly different. CircuitVQ-based synthesis assumes NAND-only gates and PI/PO constraints; extending to richer libraries such as AOI/OAI or XOR would require retraining and possibly modified encoders and decoders, and the reported metrics emphasize functional equivalence, DAS steps, and gate count rather than power, delay, or area (Wu et al., 18 Feb 2025). Circuit Transformer’s exact mask scales poorly because the full truth table gives $A \in \{0,1\}^{N \times N}$59 constraints and tri-valued propagation per candidate is $A \in \{0,1\}^{N \times N}$60, which is why the reported setting is restricted to $A \in \{0,1\}^{N \times N}$61 (Li et al., 2024). CircuitFormer’s CKT fixes vocabulary size and strips parameter values for topology-only compression metrics, so rare motifs may fall back to primitive sequences and parameter-heavy analog design remains outside the tokenizer’s main abstraction (Islam et al., 7 May 2026).

Taken together, these results suggest that circuit tokenization is becoming a principal interface between symbolic structure and neural generation. The open directions named in the literature are consistent with that interpretation: richer gate vocabularies and multi-objective optimization for logic synthesis, SAT- or BDD-backed masks for larger-$A \in \{0,1\}^{N \times N}$62 exact generation, dynamic vocabularies and semantic constraints for analog motifs, and tighter integration with broader EDA flows such as rewriting, technology mapping, timing closure, and simulation feedback (Wu et al., 18 Feb 2025, Li et al., 2024, Islam et al., 7 May 2026).