Shortcut Automata Overview

• Shortcut Automata is a polysemous term that covers methods for compressing state progression in automata, including sublinear Transformer simulations, τ-closure in semantics, and optimized reachability graphs.
• In Transformer simulation, shortcuts enable exact finite-state dynamics using a binary tree composition of transition functions, achieving sublinear depth and over 99% accuracy on varied semiautomata.
• Graph-theoretic and group-theoretic approaches leverage shortcut techniques such as Steiner states and geodesic automatic structures to create compact representations of complex state transitions.

“Shortcut automata” (Editor's term) does not designate a single standard object in the current literature. Across several arXiv threads, it refers to constructions in which automata-theoretic behavior is compressed, reparameterized, or bypassed: shallow Transformers can simulate a length-$T$ automaton in depth $o(T)$; finite automata extracted from structural operational semantics can be simplified by $\tau$-closure to remove silent steps; automatic-group theory uses automata to select shortlex or geodesic representatives and to enforce fellow-traveler constraints; and directed transition graphs can be augmented with helper states or edges so that reachable states become close in hop distance (Liu et al., 2022, Baklanova et al., 2014, Bartholdi et al., 2010, Bernstein et al., 28 Oct 2025).

1. Terminological scope and principal meanings

The expression is best treated as a polysemous umbrella rather than as a settled term of art. In the available literature, four distinct but related meanings are especially relevant.

Setting	What is “shortcut”	Role of automata
Transformer simulation	Sublinear-depth replication of recurrent automaton dynamics	Automata are the target computation
Operational semantics	Elimination of silent or stuttering transitions by $\tau$-closure	Automata are extracted and quotiented
Automatic groups	Shortlex/geodesic normal forms and controlled Cayley-graph paths	Automata recognize representatives and multipliers
Reachability compression	Added edges or Steiner states reduce hop diameter	Automata are viewed as transition graphs

This suggests a family resemblance: each construction replaces explicit step-by-step evolution by a more compact representation of state progression, path structure, or control flow. The term is nonetheless non-standard. In particular, the chapter “Groups defined by automata” does not use the exact phrase “shortcut automata”; there the closest standard notion is automatic groups, not automata groups in the self-similar sense (Bartholdi et al., 2010).

2. Shortcuts to finite-state automata in Transformers

The most literal modern use arises in “Transformers Learn Shortcuts to Automata” (Liu et al., 2022). A semiautomaton is written as

$\mathcal A=(Q,\Sigma,\delta),$

with sequential dynamics

$q_t=\delta(q_{t-1},\sigma_t).$

The simulation task is to output the full state trajectory $(q_1,\dots,q_T)$ on an input sequence of length $T$. In that setting, a shortcut is a family of simulators $\{f_T\}_{T\ge 1}$ whose depth satisfies

$D(f_T)\le o(T),$

so the model exactly reproduces the automaton’s behavior without unrolling all $o(T)$0 recurrent updates (Liu et al., 2022).

The paper’s central constructive result is that such shortcuts always exist for finite-state dynamics. For any semiautomaton $o(T)$1, initial state $o(T)$2, and sequence length $o(T)$3, there is a depth-$o(T)$4 Transformer that continuously simulates $o(T)$5, with embedding dimension $o(T)$6, MLP width $o(T)$7, $o(T)$8 heads of embedding dimension $o(T)$9, and a $\tau$0-layer MLP (Liu et al., 2022). The construction is a parallel prefix computation over the automaton’s transformation semigroup: instead of storing only the current state, the model composes the per-symbol maps $\tau$1 in a binary-tree schedule.

The same work defines continuous simulation by requiring encoding and decoding maps

$\tau$2

such that $\tau$3 simulates the automaton. This is crucial because the learned shortcut need not resemble a recurrent state machine internally; it may encode transition functions, counters, or partial compositions in a continuous workspace while preserving the exact discrete input-output relation (Liu et al., 2022).

3. Algebraic structure, constant-depth simulators, and brittleness

The shortcut perspective becomes sharper once the automaton’s algebraic structure is exposed. The same paper shows that $\tau$4-depth simulators are not exceptional: for solvable semiautomata, there is a Transformer of depth $\tau$5 that continuously simulates $\tau$6, so the depth is independent of $\tau$7 for any fixed automaton (Liu et al., 2022). The analysis uses tools from Krohn-Rhodes theory and circuit complexity. Modular counters and memory automata serve as primitive modules: the mod-$\tau$8 counter is implemented by attention-based prefix summation followed by an MLP computing modulo $\tau$9, while the memory semiautomaton is implemented by attention that retrieves the most recent non-$\tau$0 symbol.

There are also stronger special-case results. For the bounded 1D random-walk or path-integration automaton $\tau$1, the paper proves a depth-$\tau$2 shortcut with embedding dimension $\tau$3, attention width $\tau$4, and MLP width $\tau$5 (Liu et al., 2022). By contrast, a non-solvable semiautomaton cannot, for sufficiently large $\tau$6, be continuously simulated by an $\tau$7-precision Transformer whose depth is independent of $\tau$8 and whose width is polynomial in $\tau$9, unless

$\mathcal A=(Q,\Sigma,\delta),$0

This yields a structural taxonomy: $\mathcal A=(Q,\Sigma,\delta),$1-depth exact simulation is universal, but $\mathcal A=(Q,\Sigma,\delta),$2-depth simulation reflects strong algebraic restrictions.

Empirically, shallow Transformers trained by standard methods do learn such shortcuts. On 19 semiautomata, including cyclic groups, dihedral groups, $\mathcal A=(Q,\Sigma,\delta),$3, $\mathcal A=(Q,\Sigma,\delta),$4, $\mathcal A=(Q,\Sigma,\delta),$5, $\mathcal A=(Q,\Sigma,\delta),$6, bounded Dyck automata, and gridworlds, the models achieved $\mathcal A=(Q,\Sigma,\delta),$7 in-distribution token accuracy for every semiautomaton considered (Liu et al., 2022). The learned solutions were, however, brittle under distribution shift, weak at length generalization, and sensitive under sparse supervision. The same paper proposes scratchpad training with recency bias as a mitigation, in order to favor faithful recurrent state propagation over shallow shortcut solutions.

4. Automata derived from operational semantics and simplified by $\mathcal A=(Q,\Sigma,\delta),$8-closure

A second meaning of shortcut automata appears in “Abstracting an operational semantics to finite automata” (Baklanova et al., 2014). The paper begins from a small-step semantics for a simple imperative language and identifies two reasons that a naïve abstraction fails: loss of execution context and semantic artifacts that induce silent or stuttering transitions. The canonical example is the sequencing rule

$\mathcal A=(Q,\Sigma,\delta),$9

which is semantically necessary but does not correspond to a meaningful control-flow action (Baklanova et al., 2014).

The remedy is a zipper-based semantics in which a control state is a syntax-tree location plus a before/after marker: $q_t=\delta(q_{t-1},\sigma_t).$0 The full semantic configuration is

$q_t=\delta(q_{t-1},\sigma_t).$1

where $q_t=\delta(q_{t-1},\sigma_t).$2 is the zipper path and $q_t=\delta(q_{t-1},\sigma_t).$3 indicates whether execution is before or after the focused statement (Baklanova et al., 2014). This preserves context that would otherwise be lost when identical substatements occur in different places.

The automaton nodes are exactly these syntactic configurations, and only assignments are treated as visible actions; structural movement is labeled $q_t=\delta(q_{t-1},\sigma_t).$4. The paper then introduces a genuine shortcut construction: $q_t=\delta(q_{t-1},\sigma_t).$5-closure with $q_t=\delta(q_{t-1},\sigma_t).$6. The closure operator is

$q_t=\delta(q_{t-1},\sigma_t).$7

and

$q_t=\delta(q_{t-1},\sigma_t).$8

The quotient automaton $q_t=\delta(q_{t-1},\sigma_t).$9 has nodes given by these closures and, as the paper states, “has no silent edges any more” (Baklanova et al., 2014).

In this semantics-to-automata setting, “shortcut” is exact: silent administrative paths are collapsed, visible edges are redirected between closure classes, and the resulting automaton simulates the original one modulo $(q_1,\dots,q_T)$0-steps. The paper proves a simulation theorem

$(q_1,\dots,q_T)$1

so the shortening is behavior-preserving in the intended sense (Baklanova et al., 2014).

5. Automatic groups as the closest standard group-theoretic analogue

In geometric group theory, the nearest standard counterpart is the theory of automatic groups. The chapter “Groups defined by automata” distinguishes two major frameworks: automatic groups, in the Epstein–Cannon–Holt–Levy–Paterson–Thurston sense, and automata groups or self-similar groups, generated by Mealy automata acting on rooted trees (Bartholdi et al., 2010). If “shortcut automata” is intended to mean automata selecting short or geodesic representatives and constraining path behavior in Cayley graphs, then the first framework is the relevant one.

An automatic structure consists of a language automaton $(q_1,\dots,q_T)$2, recognizing a regular language of normal forms, together with multiplier automata $(q_1,\dots,q_T)$3, one for each generator $(q_1,\dots,q_T)$4, recognizing padded pairs of normal forms that differ by right multiplication by $(q_1,\dots,q_T)$5 (Bartholdi et al., 2010). The chapter emphasizes that one may choose shortlex representatives, yielding an automatic structure with uniqueness, and may further require the language to consist of minimal-length representatives, producing a geodesic automatic structure. These are the chapter’s closest formal embodiments of a shortcut idea: automata select canonical representatives that are controlled by length and local geometry.

The geometric core is the fellow-traveler property. The chapter states that

$(q_1,\dots,q_T)$6

and a constant $(q_1,\dots,q_T)$7 such that whenever $(q_1,\dots,q_T)$8 satisfy

$(q_1,\dots,q_T)$9

the corresponding Cayley-graph paths $T$0 $T$1-fellow-travel (Bartholdi et al., 2010). In hyperbolic groups, finite cone types and geodesic fellow-traveling yield geodesic automatic structures. The chapter also discusses Dehn-type shortening algorithms, cone types, combings, bicombings, and asynchronous automaticity, all of which concern controlled path systems rather than tree automorphisms.

This framework must be separated from automata groups in the self-similar sense. There, finite transducers define the group elements themselves as automorphisms or transformations of the rooted tree $T$2. That theory studies contraction, sections, branch properties, growth, amenability, and related phenomena; it is not primarily about shortcutting words or paths in a Cayley graph (Bartholdi et al., 2010).

6. Shortcut learning as a broader mechanistic lens

Outside automata theory proper, recent shortcut-learning work provides a broader mechanistic vocabulary for understanding why compact, brittle procedures are learned at all. “On the Foundations of Shortcut Learning” separates predictivity from availability and measures shortcut bias as excess reliance on the shortcut feature relative to the optimal classifier: $T$3 It reports that linear models are relatively unbiased, whereas introducing a single hidden layer with ReLU or Tanh units yields a bias, and it gives an NTK-based account of why nonlinear architectures privilege more available features (Hermann et al., 2023).

“Which Shortcut Solution Do Question Answering Models Prefer to Learn?” studies shortcut solutions as dataset-supported heuristic regimes. It defines deterministic shortcut and anti-shortcut subsets $T$4 and $T$5, shows that extractive QA models prefer answer-position shortcuts while multiple-choice QA models ultimately prefer word-label correlations, and argues that the more learnable a shortcut is, the flatter and deeper the loss landscape is around the shortcut solution in parameter space (Shinoda et al., 2022). This is not an automata model, but it is compatible with an automaton-like view in which training moves among discrete heuristic regimes.

“Mitigating Shortcut Learning with InterpoLated Learning” is similarly non-automata-theoretic, but it treats shortcuts as simple, highly prevalent, easy-to-fit feature-label mappings. Its InterpoLL method interpolates hidden representations of majority examples with same-class minority examples to weaken shortcut cues, and the paper reports improved minority generalization across encoder, encoder-decoder, and decoder-only architectures without compromising accuracy on majority examples (Korakakis et al., 7 Jul 2025). A plausible implication is that modern “shortcut automata” discussions often borrow their intuition from this broader literature on learned heuristic routines, even when the formal object is no longer a finite automaton.

7. Reachability shortcuts, Steiner states, and transition systems

A graph-theoretic interpretation appears in “Reviving Thorup’s Shortcut Conjecture” (Bernstein et al., 28 Oct 2025). If an automaton or transition system is viewed simply as a directed graph, then a shortcut construction augments it with new edges or new auxiliary states so that all reachable pairs become close in hop distance while preserving reachability among the original states. Thorup’s original edge-only conjecture asked whether one could add near-linearly many transitive-closure edges and reduce directed distances to $T$6; Hesse refuted that version. The revised Steiner Shortcut Conjecture allows new vertices $T$7 and asks for a graph

$T$8

that preserves reachability on $T$9, has $\{f_T\}_{T\ge 1}$0, and yields $\{f_T\}_{T\ge 1}$1 diameter on all reachable pairs (Bernstein et al., 28 Oct 2025).

The paper’s positive results show that many classical lower bounds collapse once Steiner vertices are allowed. For graphs $\{f_T\}_{T\ge 1}$2, it proves the existence of a $\{f_T\}_{T\ge 1}$3-thick Steiner shortcut graph with

$\{f_T\}_{T\ge 1}$4

such that every critical pair has

$\{f_T\}_{T\ge 1}$5

(Bernstein et al., 28 Oct 2025). In automata language, auxiliary states can factor a long structured transition family into a short sequence of reusable intermediate summaries.

The negative results are equally important. A Steiner shortcut graph has thickness $\{f_T\}_{T\ge 1}$6 if every path contains at most $\{f_T\}_{T\ge 1}$7 consecutive Steiner vertices. The paper proves that for every $\{f_T\}_{T\ge 1}$8, there exists an $\{f_T\}_{T\ge 1}$9-edge graph such that every $D(f_T)\le o(T),$0-thick Steiner shortcut graph either has diameter $D(f_T)\le o(T),$1 or $D(f_T)\le o(T),$2 edges; in particular, ideal near-linear-size, subpolynomial-diameter shortcuts are impossible when

$D(f_T)\le o(T),$3

(Bernstein et al., 28 Oct 2025). For automata viewed as unlabeled transition systems, this means that shallow helper-state gadgetizations cannot provide universal compression.

The transfer to automata is exact only for raw reachability. The paper explicitly notes that such shortcuts do not automatically preserve edge labels, path labels or accepted words, weights, acceptance structure, bisimulation or trace equivalence, or path counts (Bernstein et al., 28 Oct 2025). Thus the graph-theoretic shortcut automaton is a compressed transition system for reachability, not a language-preserving automaton transformation in the formal-language sense.