proc2vec: Kernel-Based Process Embedding

• proc2vec is a kernel-based embedding that converts MES process tokens into dense vectors, capturing process similarities via a string kernel.
• It employs eigendecomposition of a similarity matrix to generate spectral embeddings, outperforming one-hot encodings in defect prediction.
• Integrated within the PTR framework through route2vec, it enables effective trajectory aggregation and process-level attribution in semiconductor manufacturing.

proc2vec is a process-level embedding method introduced within the Partial Trajectory Regression (PTR) framework for wafer defect root cause analysis. In that setting, it maps discrete, heterogeneous process steps in semiconductor manufacturing into a shared Euclidean space, so that variable-length wafer trajectories can be modeled by a downstream sequence representation, route2vec, and then used for defect prediction and process-level attribution (Miyaguchi et al., 27 Jul 2025). Unlike word2vec-style models based on context prediction, proc2vec in this formulation is a kernel-based embedding derived from string representations of Manufacturing Execution System (MES) attributes and an eigendecomposition of the resulting similarity matrix.

1. Definition and problem setting

proc2vec addresses a representation problem that arises in wafer fabrication: process trajectories have variable lengths, span a large number of heterogeneous physical processes, and exhibit route variability caused by factors such as rework operations and random process waiting times (Miyaguchi et al., 27 Jul 2025). Standard vector-based regression models assume fixed-length feature vectors and do not directly accommodate sequences of heterogeneous steps. At the same time, one-hot encodings of process IDs or recipes treat each variant as fully independent, which is sample-inefficient when the effective sample size per unique process or recipe condition is small.

Within PTR, proc2vec is defined as the module that converts each process step into a dense vector $x_i \in \mathbb{R}^D$. The embedded objects are not complete routes but individual process tokens constructed from high-level MES attributes such as equipment ID, recipe ID, tool type, photo layer ID, and route ID. The purpose of the embedding is to place all process steps in a common vector space where similar tools and recipes are close together, enabling downstream sequence aggregation and attribution across routes, tools, and recipes.

The motivation is specifically tied to semiconductor manufacturing. Tool and recipe identifiers often contain meaningful substrings that encode equipment type, chamber number, route code, or layer name. proc2vec exploits this structure through a string kernel, thereby sharing statistical strength across related processes without requiring large numbers of observations for every exact process configuration.

2. Token construction and kernel embedding

The starting point of proc2vec is a synthetic tokenization of each process step. MES-level attributes are concatenated into a single string token of the form

$$\text{process token} = \text{eqp} \oplus \text{recipe} \oplus \text{tool\_type} \oplus \text{photo\_layer} \oplus \text{route} \oplus \cdots$$

where $\oplus$ denotes concatenation with a separator (Miyaguchi et al., 27 Jul 2025). Each unique token corresponds to one distinct process in the dataset, and the collection of unique tokens defines a vocabulary of size $V_d$.

Similarity between tokens is computed using a custom variant of the substring kernel. This yields a kernel matrix

$$K \in \mathbb{R}^{V_d \times V_d}, \qquad K_{i,j} = k(\text{token}_i,\text{token}_j),$$

where $k(\cdot,\cdot)$ measures shared substring structure between process tokens. The embedding is then obtained by eigendecomposition of $K$. If $\lambda_k$ is the $k$-th largest eigenvalue and $v^{(k)}$ its corresponding eigenvector, the coordinates of token $$\text{process token} = \text{eqp} \oplus \text{recipe} \oplus \text{tool\_type} \oplus \text{photo\_layer} \oplus \text{route} \oplus \cdots$$0 are

$$\text{process token} = \text{eqp} \oplus \text{recipe} \oplus \text{tool\_type} \oplus \text{photo\_layer} \oplus \text{route} \oplus \cdots$$1

This construction makes proc2vec a kernel PCA or kernel multidimensional scaling style embedding rather than a predictive neural model. There is no context window, no negative sampling, and no gradient-based training stage for the embedding itself. The geometry is induced by the string kernel, and the coordinates are recovered spectrally. The embedding dimension $$\text{process token} = \text{eqp} \oplus \text{recipe} \oplus \text{tool\_type} \oplus \text{photo\_layer} \oplus \text{route} \oplus \cdots$$2 is a hyperparameter; the exact value is not specified in the reported summary.

3. Position within route2vec and Partial Trajectory Regression

proc2vec is the first module in a three-part PTR pipeline: an embedding module, a regression module, and an attribution module (Miyaguchi et al., 27 Jul 2025). The pipeline can be summarized as

$$\text{process token} = \text{eqp} \oplus \text{recipe} \oplus \text{tool\_type} \oplus \text{photo\_layer} \oplus \text{route} \oplus \cdots$$3

where $$\text{process token} = \text{eqp} \oplus \text{recipe} \oplus \text{tool\_type} \oplus \text{photo\_layer} \oplus \text{route} \oplus \cdots$$4 is the vector representation of process step $$\text{process token} = \text{eqp} \oplus \text{recipe} \oplus \text{tool\_type} \oplus \text{photo\_layer} \oplus \text{route} \oplus \cdots$$5, $$\text{process token} = \text{eqp} \oplus \text{recipe} \oplus \text{tool\_type} \oplus \text{photo\_layer} \oplus \text{route} \oplus \cdots$$6 is the embedding of the partial trajectory up to step $$\text{process token} = \text{eqp} \oplus \text{recipe} \oplus \text{tool\_type} \oplus \text{photo\_layer} \oplus \text{route} \oplus \cdots$$7, and $$\text{process token} = \text{eqp} \oplus \text{recipe} \oplus \text{tool\_type} \oplus \text{photo\_layer} \oplus \text{route} \oplus \cdots$$8 predicts defect density.

A wafer trajectory is represented after proc2vec as a sequence

$$\text{process token} = \text{eqp} \oplus \text{recipe} \oplus \text{tool\_type} \oplus \text{photo\_layer} \oplus \text{route} \oplus \cdots$$9

with timestamps $\oplus$0. route2vec then performs the sequence aggregation through a constrained recurrent update,

$\oplus$1

where the temporal weighting $\oplus$2 is set to $\oplus$3 in the experiments. The final trajectory representation is $\oplus$4 for wafer $\oplus$5.

The downstream regression function is

$\oplus$6

although in the reported experiments the MLP is a linear layer with no hidden units. The architecture is intentionally constrained: proc2vec and the route2vec cell remain fixed, and “Only the projection layer is learnable in this architecture to ensure numerical stability under limited sample sizes” (Miyaguchi et al., 27 Jul 2025). This design places proc2vec in a foundational role: it provides the frozen process vectors that every subsequent prediction and attribution depends on.

4. Attribution semantics and counterfactual interpretation

The attribution mechanism in PTR is based on comparisons between partial trajectories. For a trajectory

$\oplus$7

route2vec produces a sequence of partial embeddings $\oplus$8, and the attribution score of process $\oplus$9 is defined as

$V_d$0

This is interpreted as the change in predicted defect density produced by including process $V_d$1 relative to the counterfactual partial trajectory that excludes it (Miyaguchi et al., 27 Jul 2025).

Because

$V_d$2

the contribution of process $V_d$3 to trajectory space is exactly a temporally weighted version of its proc2vec embedding. The attribution can therefore be written as

$V_d$4

which makes proc2vec the immediate source of the directional change in state associated with a process step.

The attribution scores satisfy an additive property: $V_d$5 This telescoping form gives the cumulative attribution a direct interpretation as reconstruction of the predicted defect density from a baseline state. It also makes the method applicable to variable-length and heterogeneous routes: every step, regardless of tool type or route length, is embedded in the same $V_d$6-dimensional space, accumulated through the same update, and scored through the same difference in predicted outcomes.

5. Empirical behavior and reported results

The reported experiments use real wafer history data from the NY CREATES fab in Albany and compare three process-step representations: a constant scalar representation, a one-hot representation, and the kernel embedding used by proc2vec (Miyaguchi et al., 27 Jul 2025). The comparison is based on the correlation coefficient between model output and actual defect density.

Representation scenario	Correlation coefficient
Constant scalar representation	0.27
One-hot representation	0.52
Kernel embedding (proc2vec)	0.61

These results indicate that one-hot encoding improves over a trivial constant representation, and that proc2vec further improves correlation by encoding similarity between related tools and recipes. With the full PTR pipeline and some additional features, the correlation between predicted and actual defect density improves to 0.87.

The paper also reports a qualitative validation through t-SNE visualization with perplexity 30. For three wafer samples, proc2vec embeddings exhibit clear clustering according to equipment type, with categories including Wet Process, RTP, Inspection, Lithography, RIE, Implant, Furnace, and CMP. This suggests that the substring-kernel embedding captures process similarity in a way that aligns with equipment classes, despite using high-level identifiers rather than low-level physical traces.

In the attribution analysis, cumulative attribution plots for a high-defect wafer display sharp jumps at two points, labeled A and B, which correspond to unusually long waiting times at specific tools. This suggests that proc2vec is not merely a compact encoding device; it is integral to a trajectory representation that supports defect prediction and localized upstream attribution.

6. Relation to other “-2vec” methods and naming conventions

proc2vec belongs to a broader family of domain-specific dense embedding methods, but its mechanism differs materially from word2vec, node2vec, and Prod2Vec-style models (Miyaguchi et al., 27 Jul 2025). In word2vec, embeddings are learned from local co-occurrence in word sequences; in node2vec, from graph neighborhoods induced by random walks. proc2vec, by contrast, derives its geometry from shared substrings in structured process identifiers and from spectral decomposition of a kernel matrix. Its analogy to those models is therefore conceptual rather than objective-level.

A related but distinct naming pattern appears in e-commerce representation learning. “Query2Prod2Vec” grounds lexical representations for product search in a latent product space learned from shopping sessions, where product embeddings are trained by “solving the same optimization problem as in classical word2vec” and query vectors are formed by averaging embeddings of clicked products (Bianchi et al., 2021). That work uses skip-gram with negative sampling over product sequences and then maps queries into the learned product space; proc2vec in PTR instead embeds process tokens through a string kernel and eigendecomposition, with no co-occurrence objective and no learned context window.

This distinction is important because the suffix “-2vec” can obscure major differences in methodology. A common misconception is to assume that proc2vec is a word2vec-style neural model applied to process trajectories. In the PTR formulation, that is not the case: proc2vec embeds individual process identities from their string attributes, whereas route2vec handles trajectory aggregation, and only the final projection layer is learnable.

7. Assumptions, limitations, and extensions

proc2vec assumes the availability of reliable high-level MES attributes such as equipment ID, recipe ID, tool type, photo layer, and route (Miyaguchi et al., 27 Jul 2025). It also assumes that string similarity in these identifiers is informative about process similarity. In many fabrication environments that assumption is plausible because naming conventions encode meaningful operational structure, but it is not guaranteed to hold universally.

Several limitations follow directly from the formulation. proc2vec does not use low-level physical traces such as pressure or temperature. It is not trained end-to-end with the defect prediction task, so the embedding remains frozen during supervised learning. It does not directly encode temporal dynamics beyond the temporal weighting introduced in route2vec. Its expressiveness is limited by the string kernel: if two processes differ substantially in physical effect but not in name structure, the embedding may place them too close together.

The paper identifies or suggests several extensions. One direction is to replace the simple linear accumulation

$V_d$7

with more expressive sequence models such as an MLP or LSTM within the route2vec cell. Another is to incorporate richer features, including process trace data, or to learn embeddings jointly with the prediction model in an end-to-end differentiable architecture. A plausible implication is that more expressive kernels or graph-based embeddings could further improve cross-process generalization, provided they preserve the sample-efficiency advantages that motivate proc2vec in the first place.