Tensorized Finite Automata

• Tensorized finite automata are models that embed discrete state machines into linear, multilinear, or tensor frameworks where states become basis vectors and transitions are represented by matrices or tensors.
• They generalize DFAs, NFAs, and WFAs by replacing discrete transition rules with linear operators, enabling efficient computation via tensor contractions and Fourier-based block diagonalization.
• This framework underpins exact neural simulations and spectral learning through low-rank Hankel tensor decompositions and modular Transformer architectures.

Tensorized finite automata are formulations in which finite-state computation is embedded into linear, multilinear, or tensorial structure: states are represented by basis vectors or more general module elements, transitions by symbol-indexed matrices or higher-order tensors, and word processing by operator composition or tensor contraction. In the canonical linearization, a finite automaton with state set $Q$ is realized on a vector space with basis $\{e_q : q \in Q\}$, so that each input symbol induces a linear operator and acceptance is recovered by a terminal readout vector. Across the literature, this perspective extends to weighted finite automata, tensor-train realizations of Hankel objects, semidirect-product and Fourier-module decompositions, TQFT functors, braid-group representations, and exact neural simulations by feedforward or Transformer architectures (Zhang, 29 Apr 2025, Li et al., 2020, Gustafson et al., 2023).

1. Canonical tensorization of finite automata

For deterministic finite automata, nondeterministic finite automata, and weighted finite automata, tensorization replaces a discrete transition rule by linear action on a finite-dimensional state space. A DFA $A = (\Sigma, Q, \delta, q_0, F)$ is embedded into a vector space $V \cong \mathbb{R}^{|Q|}$ or $\mathbb{C}^{|Q|}$ with basis $\{e_q : q \in Q\}$. For each symbol $a \in \Sigma$, one defines a linear operator $T_a \in \mathrm{End}(V)$ by $T_a e_q = e_{\delta(q,a)}$, sets $v_0 = e_{q_0}$, and uses the readout vector $\{e_q : q \in Q\}$0. The run on $\{e_q : q \in Q\}$1 is then

$\{e_q : q \in Q\}$2

This identifies a DFA with a linear finite-state machine. In a WFA, the same pattern becomes

$\{e_q : q \in Q\}$3

with $\{e_q : q \in Q\}$4 and symbol matrices $\{e_q : q \in Q\}$5 (Zhang, 29 Apr 2025, Li et al., 2020).

The same computation can be written as contraction of a third-order transition tensor. For a DFA, one may form $\{e_q : q \in Q\}$6 with

$\{e_q : q \in Q\}$7

so that processing a word is a chain of tensor contractions equivalent to multiplying transition matrices. In the WFA setting, stacking the matrices $\{e_q : q \in Q\}$8 into a transition tensor produces a tensor-network description in which each symbol fixes one slice, and the contraction of the resulting one-dimensional chain yields the scalar output. This is the basic sense in which automata become “tensorized”: the transition structure is not merely encoded numerically, but reorganized as a compositional multilinear object (Zhang, 29 Apr 2025, Li et al., 2020).

Tensorization also clarifies the relation between deterministic, nondeterministic, and weighted semantics. In the deterministic case, the state vector remains a basis vector. In the nondeterministic case, active-state sets can be encoded by binary vectors and propagated by linear maps followed by thresholding. In the weighted case, the same operator chain computes a rational or real-valued series. A plausible implication is that many apparently distinct automaton models differ less by their state-update architecture than by the semiring, module, or thresholding rule imposed on an otherwise similar tensorized core.

2. Syntactic monoids, semidirect products, and Fourier modules

A central algebraic refinement of tensorized finite automata proceeds through the syntactic monoid. For a language $\{e_q : q \in Q\}$9, the syntactic congruence induces the syntactic monoid $A = (\Sigma, Q, \delta, q_0, F)$0, and recognizing $A = (\Sigma, Q, \delta, q_0, F)$1 amounts to a morphism $A = (\Sigma, Q, \delta, q_0, F)$2 together with a distinguished subset $A = (\Sigma, Q, \delta, q_0, F)$3. In tensorized form, the monoid acts linearly through a representation $A = (\Sigma, Q, \delta, q_0, F)$4, and one may study $A = (\Sigma, Q, \delta, q_0, F)$5 as a module over $A = (\Sigma, Q, \delta, q_0, F)$6. This reframes automaton structure as representation theory of the transition monoid rather than only as graph dynamics (Zhang, 29 Apr 2025).

The semidirect-product setting makes this especially explicit. Under the structural hypothesis $A = (\Sigma, Q, \delta, q_0, F)$7, with $A = (\Sigma, Q, \delta, q_0, F)$8 a finite additive monoid and $A = (\Sigma, Q, \delta, q_0, F)$9 a finite group acting linearly on $V \cong \mathbb{R}^{|Q|}$0, multiplication takes the form

$V \cong \mathbb{R}^{|Q|}$1

In bases aligned with this decomposition, automaton transitions can be implemented as block operators that separate “memory” and “control.” For abelian $V \cong \mathbb{R}^{|Q|}$2, the discrete Fourier transform diagonalizes additive translations; the $V \cong \mathbb{R}^{|Q|}$3-action becomes a permutation of frequency coordinates. Concretely, in modular affine systems over $V \cong \mathbb{R}^{|Q|}$4, maps $V \cong \mathbb{R}^{|Q|}$5 become $V \cong \mathbb{R}^{|Q|}$6 in the Fourier basis, with $V \cong \mathbb{R}^{|Q|}$7 diagonal and $V \cong \mathbb{R}^{|Q|}$8 a permutation matrix on frequencies (Zhang, 29 Apr 2025).

This block structure yields both structural and algorithmic consequences. Dense $V \cong \mathbb{R}^{|Q|}$9 transitions may reduce to diagonal or permutation-diagonal form, turning $\mathbb{C}^{|Q|}$0 multiplies per step into $\mathbb{C}^{|Q|}$1 or even $\mathbb{C}^{|Q|}$2 per block when characters are one-dimensional. The paper explicitly connects this to shallow Transformer implementations: Fourier blocks align naturally with attention heads, and semidirect composition admits a parallel prefix computation. The resulting viewpoint does not claim that every automaton admits such a decomposition; rather, it identifies a class of transition monoids for which tensorization exposes latent parallelism and exact shallow realizations (Zhang, 29 Apr 2025).

A common misconception is that tensorization here merely means replacing symbolic states by dense continuous embeddings. In this line of work, the significant step is instead basis selection: by choosing a basis adapted to monoid structure, especially a Fourier basis on abelian memory, one obtains invariant subspaces and sparse block forms that can change computational depth without changing language semantics. The limitation is equally explicit: for arbitrary finite monoids, $\mathbb{C}^{|Q|}$3 need not be semisimple, so complete block diagonalization may fail (Zhang, 29 Apr 2025).

3. Tensor networks, Hankel structure, and spectral learning

A second major strand identifies tensorized finite automata with tensor networks, especially tensor trains or matrix product states. For a WFA with parameters $\mathbb{C}^{|Q|}$4, the order-$\mathbb{C}^{|Q|}$5 Hankel tensor

$\mathbb{C}^{|Q|}$6

admits a TT decomposition of rank at most $\mathbb{C}^{|Q|}$7, where $\mathbb{C}^{|Q|}$8 is the number of WFA states. Explicit TT cores are obtained directly from WFA parameters:

$\mathbb{C}^{|Q|}$9

The contraction of these cores reproduces the WFA value exactly, so the TT structure is an identity for recognizable functions rather than an approximation heuristic (Li et al., 2020).

This tensor-network viewpoint is tightly coupled to classical Hankel theory. If $\{e_q : q \in Q\}$0 is the Hankel matrix of a recognizable series, then for a minimal WFA with $\{e_q : q \in Q\}$1 states one has $\{e_q : q \in Q\}$2, and $\{e_q : q \in Q\}$3 with

$\{e_q : q \in Q\}$4

The TT perspective refines this by showing that finite Hankel blocks are matricizations of low-TT-rank tensors. Consequently, the expensive matrix operations of spectral learning can be carried out in TT form without materializing exponentially large Hankel blocks (Li et al., 2020).

The computational payoff is explicit. In the TT formulation, pseudoinverses and matrix products required by spectral learning can be performed with cost

$\{e_q : q \in Q\}$5

whereas the classical matrix pipeline scales as

$\{e_q : q \in Q\}$6

The paper further proves that, under exact Hankel information and a complete basis, the TT spectral algorithm reconstructs the minimal WFA exactly. It also provides an almost-sure completeness statement for length-$\{e_q : q \in Q\}$7 bases when $\{e_q : q \in Q\}$8, together with a blank-symbol workaround when fixed-length bases are not complete (Li et al., 2020).

The same article establishes a size-preserving expressive equivalence between WFAs and linear second-order RNNs on one-hot inputs. A WFA transition tensor becomes the recurrent bilinear core $\{e_q : q \in Q\}$9 of a linear 2-RNN, and conversely any such linear 2-RNN induces a WFA. This places tensorized automata, tensor trains, and bilinear recurrent models inside a single algebraic framework. The continuous-input extension goes further by recovering minimal linear 2-RNNs from finite Hankel tensors via least-squares estimation and TT-based recovery methods such as TIHT, TT-ALS, and TT-SGD, with noiseless exact recovery under stated rank and sampling conditions (Li et al., 2020).

4. Exact neural realizations

Tensorized automata have also been used as compilation targets for exact neural architectures. In one line of work, an $a \in \Sigma$0-NFA is encoded by binary active-state vectors $a \in \Sigma$1, symbol-conditioned matrices $a \in \Sigma$2, and an $a \in \Sigma$3-transition matrix $a \in \Sigma$4. Nondeterministic branching is computed by $a \in \Sigma$5 followed by exact Booleanization

$a \in \Sigma$6

and $a \in \Sigma$7-closure is implemented as

$a \in \Sigma$8

The resulting feedforward ReLU simulator has three macro-layers, width $a \in \Sigma$9, and depth linear in input length, while remaining strictly feedforward and exact rather than recurrent or approximate (Dhayalkar, 30 May 2025).

For DFAs, an explicit feedforward construction uses one-hot state embeddings and one-hot symbol embeddings. Transition blocks are implemented by two-layer ReLU modules whose hidden units correspond to state-symbol pairs and realize exact “AND” gates on the concatenated input. The paper also gives a compressed threshold-network variant with state codes of dimension $T_a \in \mathrm{End}(V)$0, together with a Myhill–Nerode class embedding preserving state distinctness in $T_a \in \mathrm{End}(V)$1 dimensions. At the same time, it states the expressivity boundary: fixed-depth, fixed-width feedforward networks cannot recognize non-regular languages such as $T_a \in \mathrm{End}(V)$2 for unbounded input lengths (Dhayalkar, 16 May 2025).

A different neural route uses logic-gated time-shared feedforward networks to simulate alternating finite automata. Here the transition structure is carried by symbol-indexed binary matrices $T_a \in \mathrm{End}(V)$3, while alternation is encoded by state-dependent biases $T_a \in \mathrm{End}(V)$4 that implement existential $T_a \in \mathrm{End}(V)$5 or universal $T_a \in \mathrm{End}(V)$6 aggregation through thresholding. The forward pass

$T_a \in \mathrm{End}(V)$7

exactly simulates AFA reachability, including instantaneous $T_a \in \mathrm{End}(V)$8-closures via monotone fixed-point iteration. Because width equals $T_a \in \mathrm{End}(V)$9, the architecture inherits the exponential succinctness of AFAs relative to NFAs for suitable language families (Dhayalkar, 20 Mar 2026).

The Transformer-based constructions are algebraically different again. For modular affine automata over $T_a e_q = e_{\delta(q,a)}$0, the Fourier basis turns each symbol action into a fixed map $T_a e_q = e_{\delta(q,a)}$1, so a constant-depth Transformer of width $T_a e_q = e_{\delta(q,a)}$2 and fixed attention structure exactly simulates the automaton on arbitrary sequence length $T_a e_q = e_{\delta(q,a)}$3. For general semidirect-product automata $T_a e_q = e_{\delta(q,a)}$4, associativity permits a Blelloch-style parallel scan, yielding a Transformer with depth $T_a e_q = e_{\delta(q,a)}$5, embedding dimension $T_a e_q = e_{\delta(q,a)}$6, attention-head width $T_a e_q = e_{\delta(q,a)}$7, and MLP width $T_a e_q = e_{\delta(q,a)}$8 (Zhang, 29 Apr 2025).

Taken together, these constructions show that “tensorized” does not pick out a single neural architecture. It denotes a compilation strategy in which automaton transitions are first expressed as structured linear operators—often sparse, blockwise, or frequency-diagonal—and only then assigned to ReLU blocks, threshold circuits, logic-gated recurrences, or attention/MLP components.

5. Categorical, braid-theoretic, and linear-storage extensions

The tensorized viewpoint has also been developed outside standard matrix-chain formulations. One categorical version packages an NFA for a regular language $T_a e_q = e_{\delta(q,a)}$9 as a one-dimensional oriented TQFT with defects. The target is the category of $v_0 = e_{q_0}$0-semimodules, the state space of a positively oriented point is the free $v_0 = e_{q_0}$1-module $v_0 = e_{q_0}$2 on the automaton state set, and each letter $v_0 = e_{q_0}$3 acts by a Boolean linear operator $v_0 = e_{q_0}$4. Word acceptance becomes the tensor contraction

$v_0 = e_{q_0}$5

while decorated circles evaluate to the Boolean trace $v_0 = e_{q_0}$6. Different NFAs for the same interval language can therefore agree on decorated intervals yet differ on decorated circles, leading to a distinct “circular language” invariant (Gustafson et al., 2023).

Another extension uses braid-group representations. In that setting, the state space is a tensor power $v_0 = e_{q_0}$7, and the braid generators act locally as

$v_0 = e_{q_0}$8

When $v_0 = e_{q_0}$9 is the Hecke or quantum-group $\{e_q : q \in Q\}$00-operator, local moves can send basis states to linear combinations, producing a nondeterministic or superposition-like finite automaton. For rack or quandle solutions, the transition is combinatorial and deterministic. The paper further studies the $\{e_q : q \in Q\}$01-invariant operator $\{e_q : q \in Q\}$02, whose eigenstates organize canonical bases of irreducible modules appearing in $\{e_q : q \in Q\}$03 (Doikou, 13 Nov 2025).

Linear-storage automata provide yet another family. Real-time vector automata maintain a rational row vector and multiply it at each step by a rational matrix, with acceptance determined by state and an equality test on one vector entry. Homing vector automata instead accept when the vector returns to its initial value, either with or without mid-run tests. These models are explicitly described as linear or tensorized views of finite automata, and their product constructions can be expressed by block-diagonal matrices on concatenated vectors or by Kronecker products on tensor products of state spaces (Salehi et al., 2013, Salehi, 2019).

A more geometric use of tensorization appears in automata encodings of piecewise polynomial functions. Rather than encoding graph points by digitwise convolution—an approach under which only linear continuously differentiable functions are finite-automaton encodable—the spline framework encodes pairs consisting of coefficients and hierarchical tensor-product B-splines. Regular languages describe hierarchical meshes, basis selection, and coefficient relations on $\{e_q : q \in Q\}$04, yielding a finite-automaton representation of regular splines with arbitrary smoothness level determined by the spline degree (Berdinsky et al., 2021).

These examples broaden the term beyond operator chains on $\{e_q : q \in Q\}$05. Tensorization may mean a symmetric monoidal functor, a braid-local action on $\{e_q : q \in Q\}$06, a linear register with homing semantics, or an automaton presentation of tensor-product spline bases. What remains common is the replacement of purely combinatorial transition graphs by compositional algebraic objects whose combination law mirrors word concatenation or geometric composition.

6. Scope, misconceptions, and open problems

The literature does not use “tensorized finite automata” for a single canonical model. Instead, it names a family of closely related constructions in which automata are embedded into linear, multilinear, or tensorial frameworks. A common misconception is that tensorization is intrinsically approximate or quantum-mechanical. In fact, several central constructions are exact: TT representations of WFAs recover the recognized function identically, ReLU and threshold networks realize NFA and DFA transitions without approximation, the Transformer constructions in the modular affine and semidirect settings are proved exact, and the TQFT packaging of NFAs is an algebraic reformulation over the Boolean semiring (Li et al., 2020, Zhang, 29 Apr 2025, Gustafson et al., 2023).

The main limitations are structural rather than conceptual. The constant-depth Transformer theorem presently covers modular affine automata over $\{e_q : q \in Q\}$07; beyond that family, the paper states that extending constant depth beyond groups or affine families remains open. For arbitrary finite monoids, failure of semisimplicity in $\{e_q : q \in Q\}$08 can obstruct complete Fourier-like block diagonalization. In the TT and spectral-learning line, guarantees rely on linear activations and exact or well-estimated Hankel objects, and extending the theory to nonlinear RNNs remains open (Zhang, 29 Apr 2025, Li et al., 2020).

Other open questions are model-specific. In the TQFT formulation, the classification of all circular languages arising from automata recognizing a fixed regular language is posed as an open problem. In the AFA simulation framework, $\{e_q : q \in Q\}$09-closure can become the main computational bottleneck for dense $\{e_q : q \in Q\}$10-graphs, and formal sample-complexity bounds for topology and gate recovery remain open. In the braid-theoretic setting, the state-space dimension grows as $\{e_q : q \in Q\}$11, and the papers themselves note that acceptance semantics there are closer to quantum or probabilistic superposition than to ordinary language recognition (Gustafson et al., 2023, Dhayalkar, 20 Mar 2026, Doikou, 13 Nov 2025).

A final theme is learnability. Several papers give constructive exact simulations, but they separate representation from optimization. One paper explicitly remarks that understanding when SGD discovers Fourier or semidirect solutions is a distinct learning-theoretic question, while the TT line studies statistically grounded recovery only for linear 2-RNNs and low-rank Hankel structure. This suggests that tensorized finite automata currently function more as a structural language for analyzing automata, representations, and exact neural compilations than as a settled theory of how such structures are routinely learned from data (Zhang, 29 Apr 2025, Li et al., 2020).