LORT: Logits Retargeting & Speech Enhancement

• LORT is an acronym representing distinct methods: Logits Retargeting for long-tailed recognition and Locally Refined Convolution and Taylor Transformer for speech enhancement.
• In long-tailed recognition, Logits Retargeting replaces one-hot targets with highly smoothed targets to balance logit magnitudes and improve classifier retraining.
• In speech enhancement, LORT integrates Taylor-based global modeling with local convolutional refinement to efficiently enhance noisy monaural signals.

LORT is an acronym used in recent machine-learning literature for at least two unrelated methods. In long-tailed recognition, LORT denotes Logits Retargeting, a classifier re-training method in the decoupled-training paradigm that replaces one-hot targets with heavily smoothed targets in order to reduce and balance logit magnitude (Lu et al., 2024). In monaural speech enhancement, LORT denotes Locally Refined Convolution and Taylor Transformer, a lightweight time-frequency-domain architecture that combines Taylor-based global modeling, spatial-channel enhancement attention, and local convolutional refinement (Wang et al., 28 Sep 2025). A plausible implication is that the acronym is domain-sensitive and should be resolved by task area rather than by string match alone.

1. Disambiguation and nomenclature

The two principal arXiv usages of the acronym are summarized below.

Expansion	Domain	Core idea
Logits Retargeting	Long-tailed recognition	Replace one-hot labels with heavily smoothed targets during classifier re-training
Locally Refined Convolution and Taylor Transformer	Monaural speech enhancement	Combine Taylor Transformer, SCEA, and LRC within a U-Net-like encoder-decoder

In the long-tailed-recognition usage, LORT is explicitly situated in the Decoupled Training paradigm, which separates representation learning from classifier re-training and re-examines the second stage under a unified feature representation from LTWB (Lu et al., 2024). In the speech-enhancement usage, LORT is a U-Net-like encoder-decoder that processes STFT magnitude and phase and explicitly decodes the two spectra independently (Wang et al., 28 Sep 2025).

The shared acronym does not indicate methodological continuity. The two methods target different data modalities, optimize different objectives, and are evaluated with different metrics. One is a classifier-head re-training rule for imbalanced image recognition; the other is a compact spectro-temporal enhancement network for noisy speech.

2. LORT as Logits Retargeting in long-tailed recognition

In long-tailed recognition, the setting is a highly imbalanced training distribution in which a few head classes have many samples while many tail classes have very few, whereas evaluation is typically performed on a balanced test set (Lu et al., 2024). The paper reconsiders the second stage of decoupled training: the backbone is frozen and only the classifier head is re-trained or finetuned.

The method is motivated by a critique of prior classifier re-training schemes. Earlier methods often changed both representation learning and classifier learning simultaneously, making it difficult to isolate the effect of classifier re-training itself. The paper also argues that strong first-stage features reduce the need for complicated classifier tricks, shifting attention to what the classifier should optimize.

LORT replaces the standard one-hot target

$$y_i = \begin{cases} 1 & i=y,\ 0 & i\neq y, \end{cases}$$

with a softened target

$\tilde{y}_i = \begin{cases} 1 - \delta + \delta/K, & i=y,\[4pt] \delta/K, & i\neq y, \end{cases}$

where $\delta \in [0,1)$ (Lu et al., 2024). The corresponding loss is

$$\mathcal{L}(\mathbf{W},\mathbf{b};f(\mathbf{x}),y) = \sum_{i=1}^K -\tilde y_i \log\left( \frac{e^{z_i}}{\sum_j e^{z_j}} \right).$$

The method is presented as a simple logits-retargeting rule that does not require prior knowledge of the number of samples per class. This distinguishes it from methods such as Balanced Softmax or logit adjustment that explicitly use class frequencies. The paper also notes that LORT is mathematically consistent with label smoothing, but uses a markedly different regime: very large smoothing values, specifically $\delta=0.98$ on CIFAR100-LT and $\delta=0.99$ on ImageNet-LT and iNaturalist2018.

3. Logit-based analysis and theoretical rationale

The analytical core of Logits Retargeting is a shift away from weight norm as the main classifier diagnostic. For class $i$, the paper defines Logits Magnitude as

$$L_i = \mathbb{E}[\mathbf{z}_{P i}] - \mathbb{E}[\mathbf{z}_{N i}],$$

where $\mathbf{z}_{P i}$ are logits of positive samples of class $i$ and $\tilde{y}_i = \begin{cases} 1 - \delta + \delta/K, & i=y,\[4pt] \delta/K, & i\neq y, \end{cases}$0 are logits of negative samples (Lu et al., 2024). The argument is that logits are a more faithful object than classifier-weight norms because prediction behavior depends on the interaction between weights, features, and bias rather than on $\tilde{y}_i = \begin{cases} 1 - \delta + \delta/K, & i=y,\[4pt] \delta/K, & i\neq y, \end{cases}$1 alone.

The paper further introduces Regularized Standard Deviation

$\tilde{y}_i = \begin{cases} 1 - \delta + \delta/K, & i=y,\[4pt] \delta/K, & i\neq y, \end{cases}$2

as an approximate invariant that is easier to analyze than Logits Magnitude itself. Empirically, $\tilde{y}_i = \begin{cases} 1 - \delta + \delta/K, & i=y,\[4pt] \delta/K, & i\neq y, \end{cases}$3 tends to grow from head to tail classes.

A central proposition is that if class-wise magnitudes are already nearly balanced, then reducing the absolute value of Logits Magnitude can decrease errors and disturbances during training. The perturbation argument is expressed through

$\tilde{y}_i = \begin{cases} 1 - \delta + \delta/K, & i=y,\[4pt] \delta/K, & i\neq y, \end{cases}$4

Under this view, smaller $\tilde{y}_i = \begin{cases} 1 - \delta + \delta/K, & i=y,\[4pt] \delta/K, & i\neq y, \end{cases}$5 implies smaller perturbation scale, which in turn reduces instability in the softmax probabilities and the gradients. Since

$\tilde{y}_i = \begin{cases} 1 - \delta + \delta/K, & i=y,\[4pt] \delta/K, & i\neq y, \end{cases}$6

stabilizing the logits is presented as a mechanism for stabilizing classifier optimization itself (Lu et al., 2024).

The paper also derives the Hessian of the cross-entropy loss with respect to logits: $\tilde{y}_i = \begin{cases} 1 - \delta + \delta/K, & i=y,\[4pt] \delta/K, & i\neq y, \end{cases}$7 and shows that for any vector $\tilde{y}_i = \begin{cases} 1 - \delta + \delta/K, & i=y,\[4pt] \delta/K, & i\neq y, \end{cases}$8,

$\tilde{y}_i = \begin{cases} 1 - \delta + \delta/K, & i=y,\[4pt] \delta/K, & i\neq y, \end{cases}$9

This establishes positive semidefiniteness and supports the convexity argument in bias space. The broader claim is not that weight norms are useless, but that logit-space quantities are more invariant and more closely tied to classifier behavior.

4. Empirical profile of Logits Retargeting

The long-tailed-recognition paper evaluates LORT on CIFAR100-LT, ImageNet-LT, and iNaturalist2018 with ResNet34, ResNeXt50, and ResNet50 backbones respectively (Lu et al., 2024). Training uses SGD with momentum $\delta \in [0,1)$0, a cosine learning-rate schedule, decoupled training, 20 classifier-finetuning epochs, standard augmentation, and the best result over 5 runs.

The reported headline results are as follows.

Dataset	LORT	Comparison reported in the paper
CIFAR100-LT	54.9 / 58.8 / 69.7	LTWB: 53.4 / 57.7 / 68.7
ImageNet-LT	54.4	LTWB: 53.9
iNaturalist2018	70.8	LTWB: 70.2

On CIFAR100-LT, the scores correspond to imbalance ratios $\delta \in [0,1)$1. The gains are described as approximately $\delta \in [0,1)$2–$\delta \in [0,1)$3 on that benchmark. On ImageNet-LT and iNaturalist2018, LORT also exceeds LTWB. The paper further states that the strongest improvement is typically on few-class categories, aligning the empirical behavior with the method’s tail-oriented motivation (Lu et al., 2024).

Ablations emphasize that the smoothing parameter is not a mild regularizer in this setting. Accuracy improves as $\delta \in [0,1)$4 increases, and the best values are very high. The method is also reported to be relatively stable across learning-rate and weight-decay settings and to improve different backbones when substituted for vanilla classifier re-training. The paper therefore presents LORT as a plug-and-play classifier-finetuning method rather than as a representation-learning method.

5. LORT as Locally Refined Convolution and Taylor Transformer

In speech enhancement, LORT is a lightweight time-frequency-domain monaural speech enhancement model that combines a Taylor Transformer variant for global modeling, a spatial-channel enhancement attention (SCEA) branch for channel and spatial interaction, a locally refined convolution (LRC) block for local detail, and a U-Net-like encoder-decoder with separate magnitude and phase decoders (Wang et al., 28 Sep 2025).

The noisy input is converted by STFT into magnitude spectrum $\delta \in [0,1)$5 and phase spectrum $\delta \in [0,1)$6, which are combined into

$\delta \in [0,1)$7

The encoder uses two convolutional layers, expands the 2 input channels to 16 channels, reduces frequency dimension by half, and applies a Dilated DenseNet with depth $\delta \in [0,1)$8 and dilation factors

$\delta \in [0,1)$9

Between encoder and decoder, LORT stacks multiple locally refined Taylor transformer blocks, with default setting

$$\mathcal{L}(\mathbf{W},\mathbf{b};f(\mathbf{x}),y) = \sum_{i=1}^K -\tilde y_i \log\left( \frac{e^{z_i}}{\sum_j e^{z_j}} \right).$$0

and uses alternating downsampling and upsampling to learn multi-resolution representations (Wang et al., 28 Sep 2025).

The Taylor multi-head self-attention (T-MSA) module approximates the exponential in MHSA with a first-order Taylor expansion. The paper contrasts the complexity of standard MHSA,

$$\mathcal{L}(\mathbf{W},\mathbf{b};f(\mathbf{x}),y) = \sum_{i=1}^K -\tilde y_i \log\left( \frac{e^{z_i}}{\sum_j e^{z_j}} \right).$$1

with that of T-MSA,

$$\mathcal{L}(\mathbf{W},\mathbf{b};f(\mathbf{x}),y) = \sum_{i=1}^K -\tilde y_i \log\left( \frac{e^{z_i}}{\sum_j e^{z_j}} \right).$$2

Because $$\mathcal{L}(\mathbf{W},\mathbf{b};f(\mathbf{x}),y) = \sum_{i=1}^K -\tilde y_i \log\left( \frac{e^{z_i}}{\sum_j e^{z_j}} \right).$$3, this is the basis for the architecture’s efficiency claim.

SCEA complements T-MSA by restoring channel interaction and spatial sensitivity. Its channel branch uses 2-D pooling over time-frequency features followed by a 1-D convolution with kernel size 3. Its spatial branch applies 1-D pooling over channels followed by a $$\mathcal{L}(\mathbf{W},\mathbf{b};f(\mathbf{x}),y) = \sum_{i=1}^K -\tilde y_i \log\left( \frac{e^{z_i}}{\sum_j e^{z_j}} \right).$$4 convolution (Wang et al., 28 Sep 2025). The LRC block then supplies local refinement through a convolutional feedforward network (CFN) and time-frequency dense local convolution (TF-DLC) with a gated unit and residual connection. In the reported configuration, TF-DLC uses depth $$\mathcal{L}(\mathbf{W},\mathbf{b};f(\mathbf{x}),y) = \sum_{i=1}^K -\tilde y_i \log\left( \frac{e^{z_i}}{\sum_j e^{z_j}} \right).$$5, dilation factor $$\mathcal{L}(\mathbf{W},\mathbf{b};f(\mathbf{x}),y) = \sum_{i=1}^K -\tilde y_i \log\left( \frac{e^{z_i}}{\sum_j e^{z_j}} \right).$$6, kernel size $$\mathcal{L}(\mathbf{W},\mathbf{b};f(\mathbf{x}),y) = \sum_{i=1}^K -\tilde y_i \log\left( \frac{e^{z_i}}{\sum_j e^{z_j}} \right).$$7, and each convolution is followed by InstanceNorm2d and PReLU.

6. Training objective, results, and ablations for speech enhancement

LORT decodes enhanced magnitude and phase independently and reconstructs speech through

$$\mathcal{L}(\mathbf{W},\mathbf{b};f(\mathbf{x}),y) = \sum_{i=1}^K -\tilde y_i \log\left( \frac{e^{z_i}}{\sum_j e^{z_j}} \right).$$8

The total loss is a composite objective,

$$\mathcal{L}(\mathbf{W},\mathbf{b};f(\mathbf{x}),y) = \sum_{i=1}^K -\tilde y_i \log\left( \frac{e^{z_i}}{\sum_j e^{z_j}} \right).$$9

with

$\delta=0.98$0

(Wang et al., 28 Sep 2025). The paper specifies complex, magnitude, phase, consistency, and adversarial components, including an inverse-wrapping phase loss and a metric-guided discriminator using normalized PESQ.

Experiments are conducted on VCTK+DEMAND and DNS Challenge 2020. Training uses 16 kHz audio, FFT length $\delta=0.98$1, window length $\delta=0.98$2, hop size $\delta=0.98$3, $\delta=0.98$4 LORT blocks, AdamW, learning rate $\delta=0.98$5, decay factor $\delta=0.98$6, and 100 epochs (Wang et al., 28 Sep 2025).

On VCTK+DEMAND, the reported metrics are:

• PESQ = 3.51
• CSIG = 4.74
• CBAK = 3.91
• COVL = 4.23
• STOI = 0.96
• 0.96M parameters
• 16.83G FLOPs

The paper states that this outperforms all models with under 1M parameters, including MUSE, S4ND U-Net, TSTNN, and DPT-FSNet, and is competitive with or slightly better than larger models such as MP-SENet, for which the reported PESQ is $\delta=0.98$7 with $\delta=0.98$8M parameters and $\delta=0.98$9G FLOPs (Wang et al., 28 Sep 2025).

On DNS Challenge 2020, LORT is reported to achieve the best or near-best performance across DNS, Babble, and Factory noise conditions. For DNS noise average, the paper gives:

• average PESQ: 3.12
• average STOI: 95.0%

Ablation results attribute measurable importance to each main component. Removing SCEA reduces PESQ from $\delta=0.99$0 to $\delta=0.99$1; removing TF-DLC produces the largest drop; replacing LRC with standard convolution also degrades performance. The paper therefore interprets the model’s gains as arising from the combination of Taylor attention, spatial-channel enhancement, and explicit local refinement.

7. Related acronyms and recurrent ambiguity around “LORT”

The string “LORT” also appears in contexts where it is plausibly intended as a reference to LoRA or to LoRA-like parameter-efficient fine-tuning methods rather than to either of the two methods above. In the discussion surrounding "TensLoRA: Tensor Alternatives for Low-Rank Adaptation", the query’s “LORT” is described as appearing to refer to LoRA or a LoRA-like low-rank adaptation method (Marmoret et al., 22 Sep 2025). A closely related observation appears in "Beyond Higher Rank: Token-wise Input-Output Projections for Efficient Low-Rank Adaptation", where “LORT” is described as most plausibly referring to LoRA / TopLoRA-style low-rank adaptation (Li et al., 27 Oct 2025).

Several near-homographic names further contribute to ambiguity. LoTR is Low Tensor Rank Weight Adaptation, a tensor decomposition generalization of LoRA in which a family of layerwise updates is modeled jointly across depth (Bershatsky et al., 2024). PreLort is Prefix-Nested LoRA for Federated Fine-Tuning under Rank Heterogeneity, which organizes LoRA dimensions into a prefix hierarchy and uses segment-wise aggregation plus prefix-nested training (Waseem et al., 14 Jun 2026). These names are distinct from both Logits Retargeting and Locally Refined Convolution and Taylor Transformer, but they increase the chance of acronym collision in PEFT-related searches.

This suggests a practical rule for literature reading: LORT is not a single established object across machine learning. In current arXiv usage, it denotes either a long-tailed-recognition classifier re-training method or a monaural speech-enhancement architecture, while in LoRA-related discussions it may arise as an apparent misspecification or near-match to other low-rank-adaptation acronyms.