InfoScale: Training-Free Scaling Methods

• InfoScale is a term for two distinct training-free techniques that mitigate information mismatch between training and inference in different modalities.
• In diffusion-based image generation, it uses plug-and-play modules (PFC, AIA, NA) to enhance resolution scalability and correct failures like detail loss and repeated structures.
• In long-context sequence modeling, InfoScale employs an entropy-preserving scaling rule for attention to counter score dilution and improve length extrapolation.

InfoScale is a name used in the 2025 arXiv literature for two distinct training-free methods. In diffusion-based image synthesis, InfoScale denotes an information-centric framework for variable-scaled image generation with pretrained latent diffusion models, designed to make models such as SD1.5, SD2.1, and SDXL operate at resolutions different from their training resolution by intervening in convolutional processing, attention-based aggregation, and initial noise (Zhang et al., 1 Sep 2025). In long-context sequence modeling, InfoScale denotes an entropy-preserving scaling rule for scaled dot-product attention under length extrapolation, designed to counter attention score dilution by replacing the vanilla $1/\sqrt{d_k}$ factor with a length-dependent multiplier derived from an information entropy invariance principle (Li et al., 15 Jan 2025). The shared name reflects a common emphasis on information mismatch between training and inference, but the two methods address different modalities, architectures, and failure modes.

1. Terminological scope and disambiguation

The term appears in at least two unrelated technical senses in the cited corpus.

Paper	Domain	Core definition
"InfoScale: Unleashing Training-free Variable-scaled Image Generation via Effective Utilization of Information" (Zhang et al., 1 Sep 2025)	Diffusion-based image generation	An information-centric framework for variable-scaled image generation
"Information Entropy Invariance: Enhancing Length Extrapolation in Attention Mechanisms" (Li et al., 15 Jan 2025)	Attention length extrapolation	An entropy-preserving scaling rule for scaled dot-product attention
"EnclaveScale: Hardware-Assisted Edge-DP for Secure Data Centre Power Telemetry" (Dang et al., 8 Jun 2026)	Secure telemetry aggregation	A separate edge-DP telemetry architecture, not an InfoScale method

In the image-generation paper, InfoScale is a multi-module inference-time framework motivated by the claim that pretrained diffusion models fail at non-training resolutions because their information processing pipeline is calibrated to the information content of the training resolution (Zhang et al., 1 Sep 2025). In the attention paper, InfoScale is a single scaling rule applied inside scaled dot-product attention, motivated by the claim that longer sequences increase the entropy of the attention distribution unless logits are rescaled (Li et al., 15 Jan 2025).

A common misconception is to treat these as a single framework spanning modalities. The sources do not support that interpretation. They are separate proposals with distinct mathematical objects: latent-image denoising, Fourier-domain compensation, and noise reshaping in one case; entropy-preserving temperature scaling for attention logits in the other. The term should also not be conflated with EnclaveScale, which addresses secure aggregation of GPU power telemetry through TDX enclaves, local differential privacy, and attestation rather than variable-scale generation or attention extrapolation (Dang et al., 8 Jun 2026).

2. InfoScale in variable-scaled image generation: problem formulation and diagnosis

In "InfoScale: Unleashing Training-free Variable-scaled Image Generation via Effective Utilization of Information" (Zhang et al., 1 Sep 2025), the problem setting is fixed-resolution pretrained latent diffusion models used at inference time on lower or higher resolutions than those seen during training. Direct inference at lower resolution tends to produce incomplete objects and poor global coherence, whereas direct inference at higher resolution often yields duplicated subjects, repeated structures, distortions, or weak local detail. The paper’s main claim is that these failures arise because the amount, spatial distribution, and frequency composition of information change with resolution, while the pretrained denoiser continues to process features, attention, and noise as if it were operating at the original scale.

The diagnosis is organized around three failure points. The first is dilated convolution. Prior higher-resolution samplers use dilation to enlarge receptive fields and reduce repeated objects, but the paper argues that dilated convolution also removes useful high-frequency information needed for detail fidelity. The layer is written as

$$f_{k}^d(h)=h\circledast\Phi_d(k),\qquad (h\circledast\Phi_d(k))(o)=\sum_{s+ d \cdot t=p}h(p)\cdot k(q),$$

where $h$ is the hidden feature, $k$ the kernel, $d$ the dilation factor, and $\Phi_d(\cdot)$ the dilation operator. The authors interpret this as effectively introducing a downsampling-then-upsampling behavior that suppresses high frequencies.

The second failure point is attention. At lower resolutions, self-attention becomes too locally focused and fails to aggregate enough global information to complete objects or scenes. At higher resolutions, attention aggregates too much redundant and repetitive information across a larger token field, encouraging repeated structures and over-response to the prompt in multiple regions. The paper studies this using information entropy computed from self-attention scores,

$H(\boldsymbol{X})=-\sum_{i=1}^n p(x_i)\log p(x_i).$

This entropy is used diagnostically to analyze how information is aggregated and transformed during denoising.

The third failure point is initial noise. The paper argues that standard i.i.d. Gaussian initialization has a spatial information distribution misaligned with non-training scales. At lower resolution, the initial noise is said to have less uniform spatial information distribution than expected at training scale, which reduces entropy and makes it harder to form a complete prompt-responsive region. At higher resolution, the noise becomes too spatially uniform over the enlarged canvas, producing multiple prompt-responsive regions that denoise into duplicated objects or repeated motifs. The paper qualitatively describes this as a multi-peak spatial response pattern.

The central interpretive framework is therefore triadic: scale-dependent information preservation, information aggregation, and information distribution. Higher-resolution generation is treated as an over-informationed regime with relatively more high-frequency content and more redundancy; lower-resolution generation is treated as an under-informationed regime with insufficient and unevenly distributed information.

3. Architecture of the diffusion-model InfoScale framework

The diffusion-model InfoScale framework introduces three plug-and-play modules applied only at inference, without retraining, finetuning, extra supervision, or additional learned parameters (Zhang et al., 1 Sep 2025). These modules are Progressive Frequency Compensation (PFC), Adaptive Information Aggregation (AIA), and Noise Adaptation (NA).

PFC is used in higher-resolution generation when the host high-resolution method employs dilated convolution. Its purpose is to compensate for high-frequency information lost by dilation by reusing information from the previous denoising step. Let $N \in \{\mathrm{unc}, c\}$ denote unconditional and conditional branches under classifier-free guidance. The paper extracts high-frequency components from the cached previous prediction and low-frequency components from the current prediction:

$$\begin{aligned} \epsilon_{t-1}^{N,h} &= \mathcal{FFT}(\epsilon_{t-1}^{N}) \odot (1-\mathcal{H}), \ \epsilon_{t}^{N,l} &= \mathcal{FFT}(\epsilon_{t}^{N}) \odot \mathcal{H}, \ \tilde{\epsilon}_{t}^{N} &= \mathcal{IFFT}\!\left(\epsilon_{t-1}^{N,h}+\epsilon_{t}^{N,l}\right). \end{aligned}$$

Here $\mathcal{H}$ is a low-pass filter with default stop frequency $$f_{k}^d(h)=h\circledast\Phi_d(k),\qquad (h\circledast\Phi_d(k))(o)=\sum_{s+ d \cdot t=p}h(p)\cdot k(q),$$0. The current timestep is trusted for coarse structure and low-frequency content, while the previous timestep donates high-frequency detail.

AIA addresses attention mismatch. Its first ingredient is Dual-Scaled Attention (DSAttn), which changes attention entropy according to resolution by modifying scaled dot-product attention with factors $$f_{k}^d(h)=h\circledast\Phi_d(k),\qquad (h\circledast\Phi_d(k))(o)=\sum_{s+ d \cdot t=p}h(p)\cdot k(q),$$1 and $$f_{k}^d(h)=h\circledast\Phi_d(k),\qquad (h\circledast\Phi_d(k))(o)=\sum_{s+ d \cdot t=p}h(p)\cdot k(q),$$2. The scale factors are

$$f_{k}^d(h)=h\circledast\Phi_d(k),\qquad (h\circledast\Phi_d(k))(o)=\sum_{s+ d \cdot t=p}h(p)\cdot k(q),$$3

with $$f_{k}^d(h)=h\circledast\Phi_d(k),\qquad (h\circledast\Phi_d(k))(o)=\sum_{s+ d \cdot t=p}h(p)\cdot k(q),$$4 by default. The paper notes that the typesetting of the DSAttn equation is slightly corrupted, but states that the intended meaning is that the scaling depends on the ratio between training-time and test-time token counts. In lower-resolution generation, DSAttn replaces original self-attention to broaden aggregation and improve completeness. In higher-resolution generation, AIA combines a DSAttn branch with the original attention branch and fuses them as

$$f_{k}^d(h)=h\circledast\Phi_d(k),\qquad (h\circledast\Phi_d(k))(o)=\sum_{s+ d \cdot t=p}h(p)\cdot k(q),$$5

where $$f_{k}^d(h)=h\circledast\Phi_d(k),\qquad (h\circledast\Phi_d(k))(o)=\sum_{s+ d \cdot t=p}h(p)\cdot k(q),$$6 is the DSAttn feature, $$f_{k}^d(h)=h\circledast\Phi_d(k),\qquad (h\circledast\Phi_d(k))(o)=\sum_{s+ d \cdot t=p}h(p)\cdot k(q),$$7 the original attention feature, and $$f_{k}^d(h)=h\circledast\Phi_d(k),\qquad (h\circledast\Phi_d(k))(o)=\sum_{s+ d \cdot t=p}h(p)\cdot k(q),$$8 and $$f_{k}^d(h)=h\circledast\Phi_d(k),\qquad (h\circledast\Phi_d(k))(o)=\sum_{s+ d \cdot t=p}h(p)\cdot k(q),$$9 are nearest-neighbor downsampling and upsampling operators. The fused feature keeps the low-frequency part of the DSAttn feature and the high-frequency part of the original attention feature.

NA reshapes the spatial information distribution of the initial latent:

$h$0

The mask $h$1 uses Gaussian weights and differs by regime. For lower-resolution generation, the paper states that the weight of $h$2 increases from the center to the surrounding “to concentrate the information in the central area,” with the intended effect of encouraging a coherent central response. For higher-resolution generation, the weight decreases from center to surrounding, suppressing excessive prompt response in the center and redistributing information more unevenly across the enlarged canvas.

The sampling pipeline is explicitly staged. First, standard Gaussian latent noise $h$3 is sampled at the target resolution. Second, NA produces $h$4. Third, denoising proceeds with the pretrained diffusion model. Fourth, lower-resolution generation replaces self-attention with DSAttn. Fifth, higher-resolution generation uses the host high-resolution inference strategy, typically including dilated convolution, and applies PFC during steps where dilation is active. Sixth, higher-resolution generation also applies AIA to fuse DSAttn and original attention features. Seventh, the sampler continues with the standard update rule.

4. Empirical behavior of the diffusion-model InfoScale

The image-generation InfoScale is demonstrated on SD1.5, SD2.1, and SDXL using 1024 random prompts from LAION-5B, with FID and KID as primary metrics, cropped variants $h$5 and $h$6, and CLIP score on SDXL tables (Zhang et al., 1 Sep 2025). The details state unseen resolution scales of $h$7, $h$8, and $h$9 relative to training resolution, while the lower-resolution table discussed in the same source reports $k$0. Concretely, the cited resolutions include $k$1, $k$2, and $k$3 for SD1.5 and SD2.1, and $k$4, $k$5, and $k$6 for SDXL.

In lower-resolution generation, the reported gains are large. At $k$7 relative scale, SD1.5 improves from FID 114.76 for direct inference and 95.09 for Attn-SF to 90.34, while KID improves from 0.031 and 0.021 to 0.017. SD2.1 improves from FID 101.79 and 88.34 to 82.93, and KID from 0.026 and 0.019 to 0.016. On SDXL, InfoScale reaches FID 71.02 and KID 0.013, is best on cropped metrics with $k$8 and $k$9, and achieves the best CLIP score at 31.04. Qualitatively, the paper shows complete content even at very low scale such as $d$0 in SDXL examples.

In higher-resolution generation, the method is primarily used as a plugin for existing high-resolution samplers. At $d$1, adding InfoScale to ScaleCrafter improves SD1.5 from FID 69.02 and $d$2 40.72 to 66.34 and 37.97, and improves SD2.1 from 64.93 and 37.70 to 62.73 and 36.47. On SDXL at $d$3, FreeScale+InfoScale improves over FreeScale from FID 51.99, KID 0.003, $d$4 60.99, $d$5 0.006, CLIP 34.23 to FID 50.79, KID 0.002, $d$6 59.50, $d$7 0.004, CLIP 34.26. At $d$8, the gains are smaller but still consistent; on SDXL, FreeScale+InfoScale improves over FreeScale from FID 60.16 to 59.74 and $d$9 from 94.20 to 92.82, with CLIP from 32.75 to 32.88.

The ablation on SD2.1 with $\Phi_d(\cdot)$0 scaling reports the following sequence: PFC only gives FID 63.99, KID 0.005, $\Phi_d(\cdot)$1 37.40, $\Phi_d(\cdot)$2 0.006; PFC + NA gives 63.48, 0.005, 37.08, 0.007; PFC + AIA gives 62.95, 0.005, 36.73, 0.006; and PFC + AIA + NA gives 62.73, 0.004, 36.47, 0.006. The qualitative ablation attributes blurred backgrounds to missing PFC, improved local detail with AIA, and removal of messy or repetitive content with NA.

The reported computational overhead is small relative to multi-stage or patch-based alternatives. For lower-resolution generation on SDXL, inference time is about 3 seconds, matching direct inference and Attn-SF, while ElasticDiffusion takes about 50 seconds. For high-resolution generation, ScaleCrafter on SDXL at $\Phi_d(\cdot)$3 goes from 38s to 40s with InfoScale, and FreeScale from 47s to 48s.

The paper also states several limitations. The analysis is described as insightful but not deeply formal; the NA mask is somewhat imprecise in the main text; the strongest SDXL numbers depend on the host sampler, especially FreeScale; and very extreme resolutions still require costly high-resolution pipelines.

5. InfoScale as entropy-preserving attention scaling

In "Information Entropy Invariance: Enhancing Length Extrapolation in Attention Mechanisms" (Li et al., 15 Jan 2025), InfoScale is an inference-time scaling rule for standard scaled dot-product attention. The motivating problem is length extrapolation failure caused by attention score dilution: as the inference sequence length $\Phi_d(\cdot)$4 grows beyond the training length $\Phi_d(\cdot)$5, the softmax denominator includes many more tokens, and normalized attention weights on important tokens shrink even if their raw affinities remain similar. The paper frames this as an increase in attention entropy and argues that preserving the entropy of the attention distribution helps preserve the model’s original focus.

The starting point is standard attention,

$\Phi_d(\cdot)$6

with coefficients

$\Phi_d(\cdot)$7

in vanilla attention. The Shannon entropy of the attention distribution for a query token is

$\Phi_d(\cdot)$8

The paper chooses $\Phi_d(\cdot)$9 so that this entropy remains approximately invariant as context length changes.

The derivation uses three assumptions: the Law of Large Numbers, embeddings lying on a $H(\boldsymbol{X})=-\sum_{i=1}^n p(x_i)\log p(x_i).$0-dimensional hypersphere of radius $H(\boldsymbol{X})=-\sum_{i=1}^n p(x_i)\log p(x_i).$1, and a Laplace approximation. This yields an entropy approximation with explicit $H(\boldsymbol{X})=-\sum_{i=1}^n p(x_i)\log p(x_i).$2 dependence and leads to a closed-form scaling rule. After calibration to the training setting and with the paper’s experimental choice $H(\boldsymbol{X})=-\sum_{i=1}^n p(x_i)\log p(x_i).$3, the final rule is

$H(\boldsymbol{X})=-\sum_{i=1}^n p(x_i)\log p(x_i).$4

Operationally, the method replaces the usual $H(\boldsymbol{X})=-\sum_{i=1}^n p(x_i)\log p(x_i).$5 factor by $H(\boldsymbol{X})=-\sum_{i=1}^n p(x_i)\log p(x_i).$6. The paper presents this as training-free and specifically designed for dot-product attention.

This distinguishes InfoScale from heuristic alternatives listed in the comparison table, including log-length scaling, Softmax Plus, and pre-softmax scaling of the YaRN style. The novelty claim is not only that attention is sharpened as length grows, but that the amount of sharpening is derived from an information entropy invariance principle rather than introduced ad hoc.

6. Empirical behavior of the attention InfoScale and related distinctions

The attention-scaling InfoScale is evaluated on the Wanjuan Patent Dataset, a long-document subset of LongData-Corpus, using GAU-$H(\boldsymbol{X})=-\sum_{i=1}^n p(x_i)\log p(x_i).$7 with top 6 layers, masked-token prediction, and metrics of perplexity (PPL) and accuracy (ACC) (Li et al., 15 Jan 2025). The presentation is explicitly inconsistent about training length: the methodology text gives a training max sequence length of 64, while many results and comparisons discuss extrapolation relative to a pretraining length of 512. The evaluation lengths are $H(\boldsymbol{X})=-\sum_{i=1}^n p(x_i)\log p(x_i).$8, and the paper emphasizes extrapolation up to 64 times the training length.

InfoScale alone improves multiple baselines. At 4096 tokens, ReRoPE changes from baseline PPL $H(\boldsymbol{X})=-\sum_{i=1}^n p(x_i)\log p(x_i).$9, ACC $N \in \{\mathrm{unc}, c\}$0 to PPL $N \in \{\mathrm{unc}, c\}$1, ACC $N \in \{\mathrm{unc}, c\}$2; YaRN(s=32) changes from PPL $N \in \{\mathrm{unc}, c\}$3, ACC $N \in \{\mathrm{unc}, c\}$4 to PPL $N \in \{\mathrm{unc}, c\}$5, ACC $N \in \{\mathrm{unc}, c\}$6; YaRN(s=16) changes from PPL $N \in \{\mathrm{unc}, c\}$7, ACC $N \in \{\mathrm{unc}, c\}$8 to PPL $N \in \{\mathrm{unc}, c\}$9, ACC $$\begin{aligned} \epsilon_{t-1}^{N,h} &= \mathcal{FFT}(\epsilon_{t-1}^{N}) \odot (1-\mathcal{H}), \ \epsilon_{t}^{N,l} &= \mathcal{FFT}(\epsilon_{t}^{N}) \odot \mathcal{H}, \ \tilde{\epsilon}_{t}^{N} &= \mathcal{IFFT}\!\left(\epsilon_{t-1}^{N,h}+\epsilon_{t}^{N,l}\right). \end{aligned}$$0; PoSE at 2048 changes from PPL $$\begin{aligned} \epsilon_{t-1}^{N,h} &= \mathcal{FFT}(\epsilon_{t-1}^{N}) \odot (1-\mathcal{H}), \ \epsilon_{t}^{N,l} &= \mathcal{FFT}(\epsilon_{t}^{N}) \odot \mathcal{H}, \ \tilde{\epsilon}_{t}^{N} &= \mathcal{IFFT}\!\left(\epsilon_{t-1}^{N,h}+\epsilon_{t}^{N,l}\right). \end{aligned}$$1, ACC $$\begin{aligned} \epsilon_{t-1}^{N,h} &= \mathcal{FFT}(\epsilon_{t-1}^{N}) \odot (1-\mathcal{H}), \ \epsilon_{t}^{N,l} &= \mathcal{FFT}(\epsilon_{t}^{N}) \odot \mathcal{H}, \ \tilde{\epsilon}_{t}^{N} &= \mathcal{IFFT}\!\left(\epsilon_{t-1}^{N,h}+\epsilon_{t}^{N,l}\right). \end{aligned}$$2 to PPL $$\begin{aligned} \epsilon_{t-1}^{N,h} &= \mathcal{FFT}(\epsilon_{t-1}^{N}) \odot (1-\mathcal{H}), \ \epsilon_{t}^{N,l} &= \mathcal{FFT}(\epsilon_{t}^{N}) \odot \mathcal{H}, \ \tilde{\epsilon}_{t}^{N} &= \mathcal{IFFT}\!\left(\epsilon_{t-1}^{N,h}+\epsilon_{t}^{N,l}\right). \end{aligned}$$3, ACC $$\begin{aligned} \epsilon_{t-1}^{N,h} &= \mathcal{FFT}(\epsilon_{t-1}^{N}) \odot (1-\mathcal{H}), \ \epsilon_{t}^{N,l} &= \mathcal{FFT}(\epsilon_{t}^{N}) \odot \mathcal{H}, \ \tilde{\epsilon}_{t}^{N} &= \mathcal{IFFT}\!\left(\epsilon_{t-1}^{N,h}+\epsilon_{t}^{N,l}\right). \end{aligned}$$4; and StreamingLLM at 4096 changes from PPL $$\begin{aligned} \epsilon_{t-1}^{N,h} &= \mathcal{FFT}(\epsilon_{t-1}^{N}) \odot (1-\mathcal{H}), \ \epsilon_{t}^{N,l} &= \mathcal{FFT}(\epsilon_{t}^{N}) \odot \mathcal{H}, \ \tilde{\epsilon}_{t}^{N} &= \mathcal{IFFT}\!\left(\epsilon_{t-1}^{N,h}+\epsilon_{t}^{N,l}\right). \end{aligned}$$5, ACC $$\begin{aligned} \epsilon_{t-1}^{N,h} &= \mathcal{FFT}(\epsilon_{t-1}^{N}) \odot (1-\mathcal{H}), \ \epsilon_{t}^{N,l} &= \mathcal{FFT}(\epsilon_{t}^{N}) \odot \mathcal{H}, \ \tilde{\epsilon}_{t}^{N} &= \mathcal{IFFT}\!\left(\epsilon_{t-1}^{N,h}+\epsilon_{t}^{N,l}\right). \end{aligned}$$6 to PPL $$\begin{aligned} \epsilon_{t-1}^{N,h} &= \mathcal{FFT}(\epsilon_{t-1}^{N}) \odot (1-\mathcal{H}), \ \epsilon_{t}^{N,l} &= \mathcal{FFT}(\epsilon_{t}^{N}) \odot \mathcal{H}, \ \tilde{\epsilon}_{t}^{N} &= \mathcal{IFFT}\!\left(\epsilon_{t-1}^{N,h}+\epsilon_{t}^{N,l}\right). \end{aligned}$$7, ACC $$\begin{aligned} \epsilon_{t-1}^{N,h} &= \mathcal{FFT}(\epsilon_{t-1}^{N}) \odot (1-\mathcal{H}), \ \epsilon_{t}^{N,l} &= \mathcal{FFT}(\epsilon_{t}^{N}) \odot \mathcal{H}, \ \tilde{\epsilon}_{t}^{N} &= \mathcal{IFFT}\!\left(\epsilon_{t-1}^{N,h}+\epsilon_{t}^{N,l}\right). \end{aligned}$$8. On the base GAU-$$\begin{aligned} \epsilon_{t-1}^{N,h} &= \mathcal{FFT}(\epsilon_{t-1}^{N}) \odot (1-\mathcal{H}), \ \epsilon_{t}^{N,l} &= \mathcal{FFT}(\epsilon_{t}^{N}) \odot \mathcal{H}, \ \tilde{\epsilon}_{t}^{N} &= \mathcal{IFFT}\!\left(\epsilon_{t-1}^{N,h}+\epsilon_{t}^{N,l}\right). \end{aligned}$$9 model, InfoScale improves 1024-token performance from PPL $\mathcal{H}$0, ACC $\mathcal{H}$1 to PPL $\mathcal{H}$2, ACC $\mathcal{H}$3, and 2048-token performance from PPL $\mathcal{H}$4, ACC $\mathcal{H}$5 to PPL $\mathcal{H}$6, ACC $\mathcal{H}$7, but still fails at 4096 with PPL $\mathcal{H}$8, ACC $\mathcal{H}$9.

The paper also introduces CosScale for scaled cosine attention and reports that the strongest results come from combining it with InfoScale. On GAU-$$f_{k}^d(h)=h\circledast\Phi_d(k),\qquad (h\circledast\Phi_d(k))(o)=\sum_{s+ d \cdot t=p}h(p)\cdot k(q),$$00 at 4096, CosScale alone gives PPL $$f_{k}^d(h)=h\circledast\Phi_d(k),\qquad (h\circledast\Phi_d(k))(o)=\sum_{s+ d \cdot t=p}h(p)\cdot k(q),$$01, ACC $$f_{k}^d(h)=h\circledast\Phi_d(k),\qquad (h\circledast\Phi_d(k))(o)=\sum_{s+ d \cdot t=p}h(p)\cdot k(q),$$02, while CosScale + InfoScale gives PPL $$f_{k}^d(h)=h\circledast\Phi_d(k),\qquad (h\circledast\Phi_d(k))(o)=\sum_{s+ d \cdot t=p}h(p)\cdot k(q),$$03, ACC $$f_{k}^d(h)=h\circledast\Phi_d(k),\qquad (h\circledast\Phi_d(k))(o)=\sum_{s+ d \cdot t=p}h(p)\cdot k(q),$$04. At 2048, CosScale gives PPL $$f_{k}^d(h)=h\circledast\Phi_d(k),\qquad (h\circledast\Phi_d(k))(o)=\sum_{s+ d \cdot t=p}h(p)\cdot k(q),$$05, ACC $$f_{k}^d(h)=h\circledast\Phi_d(k),\qquad (h\circledast\Phi_d(k))(o)=\sum_{s+ d \cdot t=p}h(p)\cdot k(q),$$06, and CosScale + InfoScale gives PPL $$f_{k}^d(h)=h\circledast\Phi_d(k),\qquad (h\circledast\Phi_d(k))(o)=\sum_{s+ d \cdot t=p}h(p)\cdot k(q),$$07, ACC $$f_{k}^d(h)=h\circledast\Phi_d(k),\qquad (h\circledast\Phi_d(k))(o)=\sum_{s+ d \cdot t=p}h(p)\cdot k(q),$$08. The paper emphasizes that cosine attention better matches the hyperspherical assumptions used in the InfoScale derivation.

The comparison with Softmax Plus is also explicit. Without CosScale, GAU-$$f_{k}^d(h)=h\circledast\Phi_d(k),\qquad (h\circledast\Phi_d(k))(o)=\sum_{s+ d \cdot t=p}h(p)\cdot k(q),$$09 + InfoScale gives PPL $$f_{k}^d(h)=h\circledast\Phi_d(k),\qquad (h\circledast\Phi_d(k))(o)=\sum_{s+ d \cdot t=p}h(p)\cdot k(q),$$10, ACC $$f_{k}^d(h)=h\circledast\Phi_d(k),\qquad (h\circledast\Phi_d(k))(o)=\sum_{s+ d \cdot t=p}h(p)\cdot k(q),$$11 at 1024 and PPL $$f_{k}^d(h)=h\circledast\Phi_d(k),\qquad (h\circledast\Phi_d(k))(o)=\sum_{s+ d \cdot t=p}h(p)\cdot k(q),$$12, ACC $$f_{k}^d(h)=h\circledast\Phi_d(k),\qquad (h\circledast\Phi_d(k))(o)=\sum_{s+ d \cdot t=p}h(p)\cdot k(q),$$13 at 2048, whereas GAU-$$f_{k}^d(h)=h\circledast\Phi_d(k),\qquad (h\circledast\Phi_d(k))(o)=\sum_{s+ d \cdot t=p}h(p)\cdot k(q),$$14 + Softmax Plus gives PPL $$f_{k}^d(h)=h\circledast\Phi_d(k),\qquad (h\circledast\Phi_d(k))(o)=\sum_{s+ d \cdot t=p}h(p)\cdot k(q),$$15, ACC $$f_{k}^d(h)=h\circledast\Phi_d(k),\qquad (h\circledast\Phi_d(k))(o)=\sum_{s+ d \cdot t=p}h(p)\cdot k(q),$$16 at 1024 and PPL $$f_{k}^d(h)=h\circledast\Phi_d(k),\qquad (h\circledast\Phi_d(k))(o)=\sum_{s+ d \cdot t=p}h(p)\cdot k(q),$$17, ACC $$f_{k}^d(h)=h\circledast\Phi_d(k),\qquad (h\circledast\Phi_d(k))(o)=\sum_{s+ d \cdot t=p}h(p)\cdot k(q),$$18 at 2048. With CosScale, the two are nearly identical at 4096: C.S. + I.S. gives PPL $$f_{k}^d(h)=h\circledast\Phi_d(k),\qquad (h\circledast\Phi_d(k))(o)=\sum_{s+ d \cdot t=p}h(p)\cdot k(q),$$19, ACC $$f_{k}^d(h)=h\circledast\Phi_d(k),\qquad (h\circledast\Phi_d(k))(o)=\sum_{s+ d \cdot t=p}h(p)\cdot k(q),$$20, and C.S. + softmax+ gives PPL $$f_{k}^d(h)=h\circledast\Phi_d(k),\qquad (h\circledast\Phi_d(k))(o)=\sum_{s+ d \cdot t=p}h(p)\cdot k(q),$$21, ACC $$f_{k}^d(h)=h\circledast\Phi_d(k),\qquad (h\circledast\Phi_d(k))(o)=\sum_{s+ d \cdot t=p}h(p)\cdot k(q),$$22.

The main limitation stated in the paper is that the derivation assumes embeddings lie on a fixed-radius hypersphere, which may not hold exactly in trained dot-product attention. This is used to explain why LM-Infinite and Windowed Attention show slight PPL degradation despite ACC gains under InfoScale, and why cosine attention strengthens the theoretical fit. A second limitation is that InfoScale alone does not fully solve extreme extrapolation on the base model.

Across the cited literature, the name therefore denotes two independent technical objects: an information-centric diffusion-model framework for variable-scaled image generation and an entropy-preserving scaling rule for attention length extrapolation. It should not be read as a unified cross-domain architecture, nor confused with EnclaveScale, which is a separate hardware-assisted edge-DP system for secure power telemetry aggregation (Dang et al., 8 Jun 2026).