Frequency Synchronization Degree (FSD)

• Frequency Synchronization Degree (FSD) is a normalized measure indicating how closely neural units or system components align to a dominant Fourier frequency for synchronization detection.
• FSD is computed via Fourier analysis of sum-conditioned activations in grokking transformers, serving as an early indicator of circuit formation that precedes generalization.
• FSD bridges diverse disciplines by linking Fourier-based synchrony in neural networks with frequency locking metrics in oscillator, optical, and communication systems.

Searching arXiv for papers on “Frequency Synchronization Degree” and related uses of FSD. Frequency Synchronization Degree (FSD) denotes, in its most explicit recent use, a normalized measure of how strongly many units align to a common dominant frequency-organized structure. In "Circuit Synchronization Precedes Generalization: Causal Evidence from Fourier Structure in Grokking Transformers," FSD is introduced as a circuit-agnostic, permutation-tested metric for Fourier circuit synchronisation in grokking transformers (Sivasankar, 11 Jun 2026). In other synchronization literatures, however, the exact term is often absent: closely related work instead quantifies the same underlying idea through carrier frequency offset estimation error, timing estimation error, global or local order parameters, effective-frequency collapse, or asymptotic convergence of hidden frequency states (Omomukuyo et al., 2018). The acronym is also overloaded; in CMB spectral-distortion measurement, FSD stands for "Frequency Space Differential," an unrelated differential-frequency observable rather than a synchronization measure (Mukherjee et al., 2018).

1. Definition, scope, and disciplinary usage

In the transformer setting, FSD is a named scalar designed to detect when a Fourier circuit has become synchronized before overt generalization appears. Its defining feature is that it requires no prior circuit knowledge: it asks only whether many neurons share the same dominant Fourier frequency in their sum-conditioned responses (Sivasankar, 11 Jun 2026).

Outside that setting, the phrase "Frequency Synchronization Degree" is usually an interpretive label rather than a standardized observable. In coherent optical synchronization, the closest operational meaning is the accuracy and robustness of carrier frequency offset (CFO) estimation and frame timing recovery, as reflected by CFO estimation error, timing estimation error, and estimator range under different OSNR and training-sequence conditions (Omomukuyo et al., 2018). In networked oscillator models, the closest quantities are order parameters such as $R$ or $r$, the fraction of locked oscillators, or direct frequency-mismatch observables such as $\Delta\Omega_{21}$ (Skardal et al., 2012). In multi-agent frequency-modulated systems, synchronization is defined asymptotically by convergence of $\sigma_i$ or $\omega_i$, not by a separate degree-like scalar (Chen, 2021).

A common misconception is that FSD names a single canonical metric across synchronization research. The arXiv record represented here indicates the opposite. One paper introduces FSD as a specific normalized statistic for Fourier circuit synchronisation in grokking (Sivasankar, 11 Jun 2026); several other papers analyze the same broad phenomenon—frequency alignment or entrainment—without defining a dedicated FSD variable (Ocampo-Espindola et al., 7 May 2025).

2. Formal construction of FSD in grokking transformers

The explicit metric FSD is built from sum-conditioned MLP activations in modular addition. For each checkpoint, input pairs $(a,b)$ are partitioned by their sum $s=(a+b)\bmod p$, and the mean MLP activation for neuron $j$ is computed as

$$A[s, j] \;=\; \frac{1}{|\mathcal{S}_s|} \sum_{(a,b):\,(a+b)\equiv s} \mathrm{GELU}\!\bigl(W_1 \mathbf{h}_{ab}\bigr)_j,$$

where $\mathbf{h}_{ab}$ is the LayerNorm-normalized residual stream at the last position. This yields a matrix $r$0, indexed by sum value and neuron (Sivasankar, 11 Jun 2026).

The activations are then centered and Fourier analyzed: $r$1 For each positive frequency $r$2, the two-sided spectral power is

$r$3

and the dominant frequency for neuron $r$4 is

$r$5

The participation fraction of frequency $r$6 is

$r$7

FSD is then defined as the normalized peak participation

$r$8

By construction, $r$9 corresponds to dominant frequencies being uniformly spread across neurons, while $\Delta\Omega_{21}$0 corresponds to perfect synchrony in which all neurons share one dominant frequency (Sivasankar, 11 Jun 2026).

The same work also defines a top-$\Delta\Omega_{21}$1 generalization for cases where more than one Fourier mode is required, and it uses a permutation test with $\Delta\Omega_{21}$2 shuffles to assess significance. The null model randomly reassigns dominant-frequency labels uniformly over $\Delta\Omega_{21}$3, and the $\Delta\Omega_{21}$4-value is computed from the fraction of null FSD values that exceed the observed one (Sivasankar, 11 Jun 2026).

3. Empirical behavior as an early indicator of grokking

Across nine modular-addition configurations—five primes $\Delta\Omega_{21}$5 and three seeds—FSD reaches a synchronized regime before grokking in every case. Synchronisation is operationalized as the first checkpoint with $\Delta\Omega_{21}$6. The reported lead times are $\Delta\Omega_{21}$7, $\Delta\Omega_{21}$8, $\Delta\Omega_{21}$9, $\sigma_i$0, $\sigma_i$1, $\sigma_i$2, $\sigma_i$3, $\sigma_i$4, and $\sigma_i$5 steps, giving a mean lead of $\sigma_i$6 steps; all nine leads are positive, with exact sign test $\sigma_i$7, bootstrap $\sigma_i$8 confidence interval $\sigma_i$9, and prime-clustered mean lead $\omega_i$0 steps with prime-clustered $\omega_i$1 confidence interval $\omega_i$2 (Sivasankar, 11 Jun 2026).

The same paper compares FSD with a restricted-logit loss baseline described as its version of Nanda et al.'s excluded loss. That baseline reconstructs activations using only the top-7 key frequencies in block-1 MLP and declares synchronisation when the restricted loss first drops below $\omega_i$3 nats. FSD synchronizes before this restricted-logit loss in all nine addition experiments. The contrast case is modular multiplication, for which FSD lags grokking by $\omega_i$4 steps; subtraction behaves like addition, with leads of $\omega_i$5 and $\omega_i$6 steps. These results delimit the metric’s domain of informativeness: it is presented not as a generic training-progress scalar, but as a marker of the Fourier-synchronization route used by addition and subtraction (Sivasankar, 11 Jun 2026).

The paper’s causal argument relies on forking training at the point where FSD has already plateaued. For add_mod97_s42, training is forked at step $\omega_i$7, when FSD has reached $\omega_i$8 but validation accuracy remains $\omega_i$9. Continuing from that checkpoint with weight decay $(a,b)$0, the stable branches exhibit strictly monotone earlier grokking as $(a,b)$1 increases: $(a,b)$2 The observed law is summarized as

$(a,b)$3

on the stable branches. The proposed theoretical form is

$(a,b)$4

which the paper fits as $(a,b)$5. Using all grokked branches $(a,b)$6, it reports $(a,b)$7, $(a,b)$8; using only the stable branches $(a,b)$9, it reports $s=(a+b)\bmod p$0, $s=(a+b)\bmod p$1. Replications across primes give $s=(a+b)\bmod p$2, $s=(a+b)\bmod p$3 for $s=(a+b)\bmod p$4; $s=(a+b)\bmod p$5, $s=(a+b)\bmod p$6 for $s=(a+b)\bmod p$7; and $s=(a+b)\bmod p$8, $s=(a+b)\bmod p$9 for $j$0 (Sivasankar, 11 Jun 2026).

4. Mechanistic interpretation and architectural dependence

In the transformer study, FSD is explicitly framed as a measure of Fourier circuit synchronisation rather than mere neuronwise similarity. The mechanistic backdrop is the Fourier identity

$j$1

which supports a frequency-specific circuit for modular arithmetic. FSD does not test whether a known hand-specified circuit is present; it tests whether dominant frequencies across neurons concentrate onto a shared mode, thereby providing a circuit-agnostic leading indicator (Sivasankar, 11 Jun 2026).

Architecture ablations show that this precursor behavior depends on a multi-block computational pathway. The reported variants are: a 1-layer standard transformer, a 2-layer attention-only model with MLPs replaced by identity, and a 2-layer MLP-only model with attention replaced by identity. The findings are: the 2-layer standard baseline groks at $j$2 with FSD synchronization at $j$3 (lead $j$4); the 1-layer standard model groks at $j$5 but FSD synchronizes at $j$6, so FSD lags by $j$7; the 2-layer attention-only model groks at $j$8 with FSD synchronization at $j$9 (lead $$A[s, j] \;=\; \frac{1}{|\mathcal{S}_s|} \sum_{(a,b):\,(a+b)\equiv s} \mathrm{GELU}\!\bigl(W_1 \mathbf{h}_{ab}\bigr)_j,$$0); and the 2-layer MLP-only model does not grok within $$A[s, j] \;=\; \frac{1}{|\mathcal{S}_s|} \sum_{(a,b):\,(a+b)\equiv s} \mathrm{GELU}\!\bigl(W_1 \mathbf{h}_{ab}\bigr)_j,$$1 steps (Sivasankar, 11 Jun 2026).

These ablations support the paper’s interpretation of grokking as a two-phase process. In the first phase, circuit formation occurs: FSD rises and Fourier rank collapses. In the second phase, circuit liberation or regularization occurs: the circuit is already complete, and weight decay removes memorization until validation accuracy jumps. This suggests that, within this experimental setting, FSD is best understood as a structural synchrony variable for an emergent algorithm rather than a generic proxy for loss reduction (Sivasankar, 11 Jun 2026).

5. FSD-like quantities in oscillator and network synchronization

In oscillator-network research, the exact term FSD is usually not introduced, but the underlying idea—how completely frequencies are entrained—is formalized through order parameters and average-frequency observables. In the degree-frequency correlated Kuramoto model,

$$A[s, j] \;=\; \frac{1}{|\mathcal{S}_s|} \sum_{(a,b):\,(a+b)\equiv s} \mathrm{GELU}\!\bigl(W_1 \mathbf{h}_{ab}\bigr)_j,$$2

the local order parameter is

$$A[s, j] \;=\; \frac{1}{|\mathcal{S}_s|} \sum_{(a,b):\,(a+b)\equiv s} \mathrm{GELU}\!\bigl(W_1 \mathbf{h}_{ab}\bigr)_j,$$3

and the global order parameter is

$$A[s, j] \;=\; \frac{1}{|\mathcal{S}_s|} \sum_{(a,b):\,(a+b)\equiv s} \mathrm{GELU}\!\bigl(W_1 \mathbf{h}_{ab}\bigr)_j,$$4

For $$A[s, j] \;=\; \frac{1}{|\mathcal{S}_s|} \sum_{(a,b):\,(a+b)\equiv s} \mathrm{GELU}\!\bigl(W_1 \mathbf{h}_{ab}\bigr)_j,$$5, the onset of the stationary synchronized state occurs at the universal threshold

$$A[s, j] \;=\; \frac{1}{|\mathcal{S}_s|} \sum_{(a,b):\,(a+b)\equiv s} \mathrm{GELU}\!\bigl(W_1 \mathbf{h}_{ab}\bigr)_j,$$6

independent of network topology, and all oscillators become phase-locked simultaneously. In this literature, an FSD-like interpretation is that $$A[s, j] \;=\; \frac{1}{|\mathcal{S}_s|} \sum_{(a,b):\,(a+b)\equiv s} \mathrm{GELU}\!\bigl(W_1 \mathbf{h}_{ab}\bigr)_j,$$7 corresponds to incoherence, the standing-wave regime corresponds to partial and oscillatory synchrony, and the stationary synchronized regime with $$A[s, j] \;=\; \frac{1}{|\mathcal{S}_s|} \sum_{(a,b):\,(a+b)\equiv s} \mathrm{GELU}\!\bigl(W_1 \mathbf{h}_{ab}\bigr)_j,$$8 corresponds to maximal synchronization because all oscillators are locked (Skardal et al., 2012).

A complementary line of work studies explosive synchronization under partial degree-frequency correlation. There the principal observables are the Kuramoto order parameter

$$A[s, j] \;=\; \frac{1}{|\mathcal{S}_s|} \sum_{(a,b):\,(a+b)\equiv s} \mathrm{GELU}\!\bigl(W_1 \mathbf{h}_{ab}\bigr)_j,$$9

effective frequencies

$\mathbf{h}_{ab}$0

and the hysteresis area $\mathbf{h}_{ab}$1 between forward and backward $\mathbf{h}_{ab}$2 curves. For nodes above a degree threshold $\mathbf{h}_{ab}$3, the correlated assignment is $\mathbf{h}_{ab}$4, while other nodes draw $\mathbf{h}_{ab}$5. The joint degree-frequency distribution is

$\mathbf{h}_{ab}$6

In BA networks, correlating only about $\mathbf{h}_{ab}$7 of the highest-degree nodes is enough to induce explosive synchronization, whereas random selection typically needs a much larger fraction and one reported BA test suggests around $\mathbf{h}_{ab}$8. In the undirected and unweighted version of the neural network of Caenorhabditis elegans, full degree-frequency correlation gives a smooth second-order transition, while restricting correlation to the $\mathbf{h}_{ab}$9 largest-degree nodes ($r$00 of nodes) produces pronounced first-order explosive synchronization with hysteresis; above threshold, nearly all oscillators collapse to a common frequency, but a small fraction $r$01 still drifts before locking at larger $r$02 (Pinto et al., 2014).

Detuning-induced synchronization provides a third formulation. In a two-population phase-oscillator model with detuning $r$03, the asymptotic average frequency of oscillator $r$04 is

$r$05

and inter-population mismatch is

$r$06

Frequency synchronization is defined by equality of these asymptotic frequencies. For one synchronized branch, the exact condition is

$r$07

For the detuning-induced branch, the approximate fold condition is

$r$08

Electrochemical oscillator experiments show that, without detuning, one population oscillates at about $r$09 Hz and the other at about $r$10 Hz; with $r$11, the populations retain their internal cluster structure but synchronize globally at about $r$12 Hz, and the measured frequency difference decreases to zero for

$r$13

This suggests an FSD interpretation based directly on whether $r$14 and on the width of the detuning interval over which that equality persists (Ocampo-Espindola et al., 7 May 2025).

6. Engineering and systems interpretations of frequency synchronization quality

In coherent optical systems, a joint frame and carrier frequency synchronization algorithm based on the fractional Fourier transform uses two discrete-time linear chirp signals as a training sequence and recovers timing offset and CFO from fractional cross-correlation peaks. Although no explicit FSD term is used, synchronization quality is evaluated by mean timing estimation error, mean CFO estimation error, and robustness with respect to TS length, actual CFO, and OSNR. Reported results include: with TS length $r$15, no timing estimation errors were observed; CFO estimation error was about $r$16 MHz in that setting; CFOs as high as $r$17 GHz could be estimated with maximum CFO estimation error around $r$18 MHz; the derived CFO range was about $r$19 GHz for the chosen parameters; and the scheme was more robust to ASE noise than the Schmidl-Cox frame synchronizer and the Zhou TS-based CFO estimator (Omomukuyo et al., 2018). In this context, an FSD-like reading corresponds to how sharply the FRFT correlation metric peaks, whether timing is recovered exactly, and how small and robust the CFO error remains.

In long-range microwave wireless synchronization, the closest FSD-like notion is wireless frequency locking between transceivers. A primary node transmits a signal modulated by its LO-derived frequency reference; the secondary node demodulates that reference through a self-mixing circuit and feeds it to a PLL so that the two oscillators are locked. The principal downstream metric is coherent gain,

$r$20

with $r$21 under perfect phase correction, together with the probability criterion

$r$22

The system demonstrates continuous high-accuracy links over a $r$23 m outdoor path for durations up to seven days. By maintaining a $r$24 mm ranging standard deviation, coherent beamforming at $r$25 GHz, $r$26 GHz, and $r$27 GHz could be supported for probabilities $r$28, $r$29, and $r$30, respectively, of achieving at least $r$31 coherent gain (Mghabghab et al., 2020). Here, synchronization degree is not a standalone index; it is inferred from the stability of the locked link and the coherence of distributed transmission.

For frequency-modulated multi-agent systems, synchronization is defined asymptotically as

$r$32

not as synchronization of the transmitted signals $r$33. The model

$r$34

is coupled to a frequency observer and a consensus loop, and convergence is established through a small-gain condition. Key bounds include

$r$35

$r$36

and the sufficient condition

$r$37

This framework supplies rigorous synchronization errors and convergence guarantees, but no normalized FSD scalar (Chen, 2021).

Low-inertia power-system analysis extends the idea further by embedding frequency synchronization in a broader complex-frequency formalism. With bus voltage written as $r$38, the complex frequency is defined as

$r$39

where the real part $r$40 is the normalized rate of change of voltage magnitude and the imaginary part is the conventional angular frequency. Synchronization requires convergence of all nodes to the same limiting complex frequency. Practical FSD-like proxies then include the subnetwork discrepancy

$r$41

the frequency convergence rate

$r$42

frequency overshoot

$r$43

and generalized inertia

$r$44

The IEEE 9-bus case study shows that buses can exhibit similar frequency trends yet differ in the real part of complex frequency, so a system that appears synchronized under frequency-only criteria may fail the stronger complex-frequency criterion (Wei et al., 15 Aug 2025).

7. Terminological ambiguity and conceptual boundaries

The acronym FSD is not unique to synchronization science. In CMB spectral-distortion measurement, FSD denotes "Frequency Space Differential," a method based on inter-frequency differences of brightness temperature. The fundamental observable is

$r$45

which, for closely spaced channels, is approximated by the frequency derivative of the relevant spectral component. In the absence of distortions, the inter-frequency difference follows the derivative of the blackbody spectrum; with $r$46- or $r$47-distortions, additional derivative signatures appear. The method is proposed as a way to measure spectral distortions without an internal blackbody calibrator (Mukherjee et al., 2018).

This terminological collision matters because it separates two unrelated uses of the same acronym. In the grokking-transformer paper, FSD is a normalized synchrony metric over dominant Fourier frequencies in neural activations (Sivasankar, 11 Jun 2026). In the CMB paper, FSD is a differential spectroscopy method (Mukherjee et al., 2018). By contrast, in optical, wireless, oscillator-network, and multi-agent synchronization papers, the phrase "Frequency Synchronization Degree" is best treated as an interpretive umbrella for observables that quantify frequency locking, synchronization accuracy, or convergence robustness rather than as the name of a universal formula (Omomukuyo et al., 2018).

Taken together, these uses show that "frequency synchronization degree" has both a narrow and a broad meaning. Narrowly, it is a specific normalized statistic for Fourier circuit synchronisation in grokking. Broadly, it denotes the extent to which a system’s components entrain in frequency, whether that extent is captured by dominant-frequency concentration, order parameters, effective-frequency collapse, CFO estimation error, coherent gain, or convergence rates. This suggests that the technical content of FSD is context-dependent: in machine learning it measures synchronization of an emergent internal representation; in physical and engineered dynamical systems it is more often reconstructed from the variables that certify locking, tracking, or common asymptotic frequency.