TFgram: Dual Applications in Audio and Language

• TFgram is a dual-use concept representing a learned time–frequency representation in audio and a syntax-aware Transformer Grammar in language modeling.
• In anomalous sound detection, TFgramNet employs trainable Conv1D layers to learn discriminative time–frequency filters that reduce false alarms.
• In language modeling, the Transformer Grammar variant leverages a recursive, stack-based attention mask to integrate syntactic structure for improved parsing and evaluation.

Searching arXiv for papers using the term “TFgram” to ground the article and resolve the terminology. TFgram is a label used in two distinct technical senses. In anomalous sound detection, it denotes a learned time–frequency representation extracted directly from raw audio by a small convolutional network, TFgramNet, and used within the one-stage supervised contrastive learning framework for unsupervised anomalous sound detection (Huang et al., 17 Sep 2025). In language modeling, an integrated description of Transformer Grammars uses “TFgram” as a label for a syntax-aware Transformer-XL variant whose typed attention mask implements recursive syntactic composition through stack and compose operations (Sartran et al., 2022). The two usages are unrelated in domain and objective, but both replace fixed or unstructured representations with learned inductive structure.

1. Terminological scope

The term has acquired an overloaded meaning because it appears in both audio representation learning and syntax-aware language modeling. The distinction is substantive rather than cosmetic: in the audio setting, TFgram is a two-dimensional representation with temporal and spectral axes; in the language-modeling setting, “TFgram” refers to a full generative model over linearized trees.

Usage	Domain	Core definition
TFgram	Anomalous sound detection	Learned time–frequency representation extracted from raw waveform via TFgramNet
TFgram (TG)	Language modeling	Transformer Grammars with a typed stack/compose attention mask

For anomalous sound detection, the representation is introduced to address frequent false alarms when handling samples of the same type from different machines, and it is evaluated on DCASE 2020 Challenge Task 2 (Huang et al., 17 Sep 2025). For language modeling, the model is evaluated on PTB and BLLIP sentence- and document-level perplexity, syntax-sensitive language modeling evaluation, and parse reranking (Sartran et al., 2022).

2. TFgram as a learned time–frequency representation for anomalous sound detection

In anomalous sound detection, TFgram is defined as a learned time–frequency representation extracted directly from the raw waveform via a small convolutional network, TFgramNet. Formally, $\mathrm{TFgram} \in \mathbb{R}^{F\times T}$ is the output of TFgramNet that maps a one-dimensional audio signal $x\in\mathbb{R}^L$ to a two-dimensional representation with $F$ “frequency” channels and $T$ time frames (Huang et al., 17 Sep 2025).

Its role is to complement or replace log-Mel. Unlike a classical spectrogram or log-Mel, which are built on a fixed short-time Fourier transform and Mel-filterbank, TFgram uses trainable one-dimensional convolutions and pooling to discover the most discriminative frequency-like filters and time-local patterns for distinguishing normal versus anomalous machine sounds. It is designed to capture machine-specific spectral patterns, including non-stationary transients, and to use global max-pooling to emphasize salient events.

This design preserves the two-axis organization associated with time–frequency analysis while removing the fixed transform assumptions of $|\mathrm{STFT}|$ and log-Mel. The reported advantages are end-to-end learning of filters tailored to distinguishing normal and anomalous machine sounds, the use of non-linear activations and pooling to emphasize high-energy transients, the injection of invariance to time shifts through global pooling, and the absence of a fixed perceptual filterbank that might discard “high-frequency” components.

3. TFgramNet extraction pipeline and mathematical formulation

The extraction pipeline begins with a single-channel waveform $x\in\mathbb{R}^L$. TFgramNet comprises a Conv1D layer with kernel size $K_1=11$, stride $s=5$, padding $p_1=\lfloor K_1/2\rfloor$, and channels $C_1=64$. It is followed by three convolutional blocks. ConvBlock$x\in\mathbb{R}^L$0 contains two Conv1D layers with kernel size $x\in\mathbb{R}^L$1, stride $x\in\mathbb{R}^L$2, padding $x\in\mathbb{R}^L$3 or $x\in\mathbb{R}^L$4, and dilation $x\in\mathbb{R}^L$5 or $x\in\mathbb{R}^L$6, each followed by BatchNorm and ReLU, then a temporal max-pool with pool size $x\in\mathbb{R}^L$7. ConvBlock$x\in\mathbb{R}^L$8 is identical to ConvBlock$x\in\mathbb{R}^L$9 and is followed by an adaptive max-pool along time to length $F$0. ConvBlock$F$1 uses $F$2 channels, the same convolutional structure, and an adaptive max-pool to $F$3 (Huang et al., 17 Sep 2025).

The first-layer feature map is given by

$F$4

for $F$5 and $F$6, with $F$7 and $F$8. Each subsequent ConvBlock applies

$F$9

with kernel $T$0, stride $T$1, and padding $T$2 or dilation. Pooling is written as

$T$3

for pool size $T$4, followed by adaptive max-pooling along time into $T$5 frames. The final representation is

$T$6

so $T$7.

Relative to spectrogram and log-Mel, the key contrast is that spectrogram uses a sliding window and FFT to yield fixed linear-frequency bins, log-Mel applies a fixed Mel-warped triangular filterbank and logarithm, and TFgram replaces these fixed transforms with trainable one-dimensional convolutions in the time domain.

4. Integration with OS-SCL and empirical behavior

Within the OS-SCL framework, TFgram features are fused or used alone as input to a lightweight embedding network, MobileFaceNet. TFgram, or the concatenated TFSTgram $T$8, is passed through MobileFaceNet to produce embeddings $T$9. A small autoencoder-style Feature Perturbation Head perturbs $|\mathrm{STFT}|$0 on the unit sphere after $|\mathrm{STFT}|$1-normalization, and the system is jointly optimized with a noisy supervised contrastive loss and a Noisy-ArcMix-augmented cross-entropy loss (Huang et al., 17 Sep 2025).

The noisy supervised contrastive loss is

$|\mathrm{STFT}|$2

According to the reported rationale, because TFgram encodes richer patterns than log-Mel alone, the embeddings separate normal IDs more distinctly under contrastive learning, reducing false alarms on similar normal samples.

The quantitative results are explicit. Using only log-Mel with OS-SCL yields average $|\mathrm{STFT}|$3, $|\mathrm{STFT}|$4, and $|\mathrm{STFT}|$5. Replacing or augmenting with TFSTgram pushes these to $|\mathrm{STFT}|$6 $|\mathrm{STFT}|$7, $|\mathrm{STFT}|$8 $|\mathrm{STFT}|$9, and $x\in\mathbb{R}^L$0 $x\in\mathbb{R}^L$1. In a direct ablation on “ArcFace only” versus “OS-SCL,” TFSTgram improves mean AUC from $x\in\mathbb{R}^L$2 to $x\in\mathbb{R}^L$3. For the Feature Perturbation Head, compression dimension $x\in\mathbb{R}^L$4 gives the best AUC of $x\in\mathbb{R}^L$5, while removing the Feature Perturbation Head drops AUC to $x\in\mathbb{R}^L$6. The abstract also reports that TFgram extracted from raw audio ultimately achieves $x\in\mathbb{R}^L$7 AUC, $x\in\mathbb{R}^L$8 pAUC, and $x\in\mathbb{R}^L$9 mAUC on DCASE 2020 Challenge Task 2.

A common misconception is to equate TFgram with a conventional spectrogram. The formulation and pipeline make clear that it is not a fixed Fourier-domain representation; it is a learned representation whose “frequency” axis emerges from trainable filters and pooling rather than from a predefined harmonic or perceptual analysis.

5. TFgram as Transformer Grammars

In a separate usage, “TFgram” denotes Transformer Grammars, a class of Transformer LLMs that combines the scalability of Transformers with recursive syntactic compositions. The model builds on the multi-segment, relative-position Transformer-XL backbone but replaces the standard causal or full-context attention mask with a typed mask that implements a stack-based recursive compose operation whenever a constituent closes (Sartran et al., 2022).

The input is a linearized action sequence

$K_1=11$0

where each $K_1=11$1 is one of four action types: an opening nonterminal $K_1=11$2, a terminal token of type $K_1=11$3, a compose closing nonterminal $K_1=11$4, or a stack closing nonterminal $K_1=11$5. A shared embedding matrix $K_1=11$6 maps each action to $K_1=11$7.

The tree-aware relative-position matrix is

$K_1=11$8

where $K_1=11$9 is the depth of token $s=5$0 in the syntax tree. The binary mask is

$s=5$1

If $s=5$2 is $s=5$3, then $s=5$4 is exactly the indices of the children of that constituent plus the opening $s=5$5, so a single composed vector can be formed summarizing the subtree. If $s=5$6 is not a compose step, then $s=5$7 is the set of all indices currently on the stack, yielding stack attention. The integrated description states that recursive composition emerges layer by layer because every attention layer uses the same mask.

The tree-to-sequence transformation duplicates every $s=5$8 into $s=5$9 followed immediately by $p_1=\lfloor K_1/2\rfloor$0. Predicting the resulting action sequence is equivalent to jointly modeling $p_1=\lfloor K_1/2\rfloor$1. When the model reaches a compose position, the self-attention layer computes

$p_1=\lfloor K_1/2\rfloor$2

which becomes the one-vector representation of the subtree. This bottleneck enforces bottom-up recursive composition because later predictions can only access a phrase through its composed vector once the interior nodes have been popped.

The Transformer-XL configuration used in the experiments is also specified: $p_1=\lfloor K_1/2\rfloor$3 layers, $p_1=\lfloor K_1/2\rfloor$4 model dimension, $p_1=\lfloor K_1/2\rfloor$5, $p_1=\lfloor K_1/2\rfloor$6 attention heads, segment length $p_1=\lfloor K_1/2\rfloor$7, memory $p_1=\lfloor K_1/2\rfloor$8, and total parameters of $p_1=\lfloor K_1/2\rfloor$9 on PTB and $C_1=64$0 on BLLIP.

6. Training, evaluation, and limitations of Transformer Grammars

Transformer Grammars are trained by maximizing the joint log-likelihood of the action sequence,

$C_1=64$1

using plain cross-entropy or next-token prediction. Because marginalizing $C_1=64$2 is intractable, test-time evaluation lower-bounds $C_1=64$3 by summing over $C_1=64$4 candidate trees proposed by a separately trained discriminative RNNG. The reported training regimen uses Adam with $C_1=64$5, $C_1=64$6, and $C_1=64$7, initial learning rate $C_1=64$8, $C_1=64$9 warmup steps, linear decay thereafter, dropout $x\in\mathbb{R}^L$00 on attention and FFN, batch size $x\in\mathbb{R}^L$01 segments per GPU with segment length $x\in\mathbb{R}^L$02, and maximum tokens per batch of approximately $x\in\mathbb{R}^L$03 (Sartran et al., 2022).

The empirical results are mixed in a principled way. Test-set means $x\in\mathbb{R}^L$04 standard deviation over $x\in\mathbb{R}^L$05 random initializations are $x\in\mathbb{R}^L$06 PTB PPL, $x\in\mathbb{R}^L$07 BLLIP sentence PPL, $x\in\mathbb{R}^L$08 BLLIP document PPL, $x\in\mathbb{R}^L$09 on the syntax-sensitive generalization suite, and $x\in\mathbb{R}^L$10 parse $x\in\mathbb{R}^L$11. In the same report, TXL (trees) attains $x\in\mathbb{R}^L$12, $x\in\mathbb{R}^L$13, $x\in\mathbb{R}^L$14, $x\in\mathbb{R}^L$15, and $x\in\mathbb{R}^L$16, while TXL (terminals) attains $x\in\mathbb{R}^L$17, $x\in\mathbb{R}^L$18, $x\in\mathbb{R}^L$19, and $x\in\mathbb{R}^L$20. The integrated description states that TG outperforms all comparably sized baselines on the syntax-sensitive generalization suite and even beats GPT-2, Gopher, and Chinchilla on that benchmark, while its strict single-vector sentence bottleneck harms document-level perplexity.

The ablation results sharpen the interpretation. Replacing gold trees with strictly left-branching, right-branching, or child-order-reversed trees catastrophically degrades TG’s validation perplexity to $x\in\mathbb{R}^L$21–$x\in\mathbb{R}^L$22 and syntax-sensitive generalization to approximately $x\in\mathbb{R}^L$23–$x\in\mathbb{R}^L$24, indicating that the mask must reflect true constituency. Dropping all relative-position signals raises perplexity by approximately $x\in\mathbb{R}^L$25–$x\in\mathbb{R}^L$26 points across models but only mildly affects syntax-sensitive generalization and parse $x\in\mathbb{R}^L$27. Regression on log-probability differences shows estimated coefficients of approximately $x\in\mathbb{R}^L$28 for high-frequency words, $x\in\mathbb{R}^L$29 for medium-frequency words, and $x\in\mathbb{R}^L$30 for low-frequency words, while copying across sentences incurs a stronger disadvantage of approximately $x\in\mathbb{R}^L$31. Probing with an MDL-based probe shows that TG and TXL (trees) encode constituency tags more saliently than TXL (terminals), that compose representations in TG are especially good at predicting the top-children of each phrase, and that TXL (trees) retains more global word-content information.

A common misconception is that stronger syntactic inductive bias must uniformly improve all language-modeling regimes. The reported document-level results argue against that interpretation: the recursive composition bottleneck improves syntax-sensitive behavior and sentence-level modeling but weakens lexical memory and cross-sentence copying. This suggests that, in the language-modeling usage of the term, “TFgram” is best understood as a model that exposes the trade-off between strict bottom-up composition and unfettered long-range attention.