DynaSpec in Imaging & Language Modeling

• DynaSpec is a dual-nature term encompassing a dynamic hyperspectral imaging dataset for video-level reconstruction and a context-aware dynamic shortlisting mechanism for speculative decoding.
• The imaging contribution provides a high-fidelity dataset with calibrated, full hyperspectral video sequences that capture temporal dynamics and support robust SCI benchmarking.
• The language modeling contribution introduces a routing-based dynamic shortlist via token clustering to reduce inference cost while preserving output exactness.

DynaSpec denotes two distinct research contributions that share the same name. In spectral imaging, DynaSpec is the first high-quality dynamic hyperspectral image dataset explicitly constructed for video-level spectral compressive imaging reconstruction, with each sample represented as a hyperspectral video sequence $X \in \mathbb{R}^{T \times H \times W \times C}$ whose frames are full hyperspectral cubes rather than pseudo-videos synthesized from still images (Cai et al., 28 Feb 2026). In large-vocabulary language modeling, DynaSpec is a context-aware dynamic shortlisting mechanism for speculative decoding that reduces the drafter’s output-head cost by routing each context to a small set of token clusters while preserving exactness because verification remains over the full target vocabulary (Zhang et al., 11 Oct 2025). The shared name therefore refers not to a single unified framework, but to two unrelated systems situated in different subfields: video-level spectral reconstruction and efficient autoregressive inference.

1. Disambiguation and conceptual scope

The imaging DynaSpec was created because existing SCI reconstruction methods are primarily image-based and therefore suffer from two limitations: the encoding process masks spatial-spectral features, creating uncertainty in reconstructing missing information from single compressed measurements, and frame-by-frame reconstruction fails to ensure temporal consistency in video perception (Cai et al., 28 Feb 2026). Its function is to provide real dynamic HSIs that make it possible to move SCI reconstruction from the image level to the video level by exploiting temporal redundancy and complementarity across adjacent frames.

The language-model DynaSpec addresses a different bottleneck. In speculative decoding, a small drafter proposes multiple tokens and a large target model verifies them once per speculation length; however, as vocabularies scale from $32\,\mathrm{k}$ to $128\,\mathrm{k}$, $152\,\mathrm{k}$, and $262\,\mathrm{k}$, the drafter’s LM head with $\mathcal{O}(|V|d)$ parameters can become a latency bottleneck, especially for very small drafters such as one-block EAGLE-style models (Zhang et al., 11 Oct 2025). DynaSpec replaces static frequency-ranked shortlists with a context-dependent dynamic shortlist built from token clusters selected by a lightweight router.

A common misconception is that the two uses of the name describe variants of a single method. The available records indicate no such connection. One is a dataset-model-benchmark ecosystem for spectral compressive imaging; the other is an inference-time vocabulary-pruning mechanism for speculative decoding.

2. DynaSpec as a dynamic hyperspectral image dataset

In the spectral-imaging literature, DynaSpec is defined as a dynamic hyperspectral image dataset designed for video-level spectral compressive imaging reconstruction (Cai et al., 28 Feb 2026). Each sequence is a 4D tensor $X \in \mathbb{R}^{T \times H \times W \times C}$. The dataset comprises 30 sequences and 300 HSIs, with each sequence containing 10 frames on average. Each frame has spatial resolution $1280 \times 1280$, the spectral range is 400–700 nm, the spectral resolution is $2\,\text{nm}$, and the number of spectral bands is

$C = \frac{700-400}{2} + 1 = 151.$

Ground-truth spectral videos are acquired using a GaiaField hyperspectral imaging system (Dualix), which is a push-broom hyperspectral camera. Because conventional line scanning would cause severe mis-registration in dynamic scenes, the dataset is acquired through a controlled frame-by-frame strategy: controlled objects and manually designed motions are used; each hyperspectral frame is acquired independently while the scene is held static; and objects are moved between frames to create dynamic sequences with realistic motion patterns (Cai et al., 28 Feb 2026). The resulting dataset is therefore a frame-scanning dataset in which sequential HSIs are captured one by one, with physical object motion between them.

Several acquisition principles are specified to ensure ground-truth fidelity. Object motion between consecutive frames is continuous and obeys physical laws, including translation, rotation, and articulated motion. Long integration times are used to reduce sensor noise and improve signal-to-noise ratio. Spectral correction is applied bandwise according to the camera’s spectral response. Illumination normalization removes the spectral properties of illumination so that the data approximates reflectance rather than raw radiance. Intensity calibration uses invariant objects in each sequence to compensate for temperature drift or long-term variation in sensor response. These steps yield HSIs with high spectral fidelity, inter-frame consistency in radiometric scale, and realistic dynamic content (Cai et al., 28 Feb 2026).

DynaSpec was positioned against limitations of previous datasets and settings. CAVE and KAIST are described as image-based; pseudo-videos obtained from cropping or sliding windows over static images emulate camera motion rather than object or articulated motion. Video datasets designed for downstream tasks may have low spectral resolution or reduced data fidelity, and are therefore inadequate for learning temporal priors for SCI reconstruction or for evaluating temporal consistency and spectral fidelity in realistic dynamic scenes (Cai et al., 28 Feb 2026). This suggests that the distinctive contribution of DynaSpec is not only dynamic content, but dynamic content calibrated to serve as reconstruction ground truth.

3. SCI simulation, PG-SVRT, and benchmarking on the imaging DynaSpec

Although the ground truth is captured by a push-broom system, SCI measurements are simulated from DynaSpec for four architectures: PMVIS, SD-CASSI, NDSSI, and DD-CASSI (Cai et al., 28 Feb 2026). For each frame $32\,\mathrm{k}$0, the unified forward model is

$32\,\mathrm{k}$1

where $32\,\mathrm{k}$2 is the encoding operator and $32\,\mathrm{k}$3 is measurement noise. Stacked over time, the video-level problem uses $32\,\mathrm{k}$4 and $32\,\mathrm{k}$5. For SD systems such as SD-CASSI and PMVIS,

$32\,\mathrm{k}$6

with $32\,\mathrm{k}$7. For DD systems such as DD-CASSI and NDSSI,

$32\,\mathrm{k}$8

with $32\,\mathrm{k}$9 because the second disperser cancels the shift.

The associated reconstruction model is PG-SVRT, a U-Net-like video transformer with three principal modules: Mask-Guided Degradation Perception, Cross-Domain Propagated Attention, and Multi-Domain Feed-Forward Network (Cai et al., 28 Feb 2026). In typical SCI experiments, 30 bands in the range 500–650 nm are selected for consistency with the real DD-CASSI prototype, so $128\,\mathrm{k}$0, and all sequences are cropped to $128\,\mathrm{k}$1 patches. PG-SVRT takes measurement sequences $128\,\mathrm{k}$2 as input and operates on a concatenation of raw measurements and degradation features.

MGDP uses the known mask pattern $128\,\mathrm{k}$3 and the system model to estimate how each spatial-spectral location is degraded. CDPA performs spatial-then-temporal attention with propagation via bridged tokens. Given input feature $128\,\mathrm{k}$4, it computes

$128\,\mathrm{k}$5

then applies spatial attention over windows with $128\,\mathrm{k}$6, $128\,\mathrm{k}$7, introduces bridged tokens with $128\,\mathrm{k}$8, and uses

$128\,\mathrm{k}$9

Its total complexity is reported as

$152\,\mathrm{k}$0

MDFFN decomposes spatiotemporal processing in a lightweight feed-forward stage. The paper reports that spatial-then-temporal ordering with value sharing outperforms temporal-then-spatial and parallel designs in ablations, especially on DynaSpec, where temporal correlation is crucial (Cai et al., 28 Feb 2026).

The benchmarking protocol combines CAVE and DynaSpec for training and KAIST plus held-out DynaSpec sequences for testing. Training uses CAVE and 25 DynaSpec sequences; testing uses KAIST and the remaining 5 DynaSpec sequences. During training, all images and sequences are cropped to $152\,\mathrm{k}$1 patches. For DynaSpec, there is a 70% chance to fix the crop location across time with step size 0, thereby preserving native temporal dynamics rather than synthesizing camera motion. The reconstruction window uses $152\,\mathrm{k}$2 frames. Optimization uses multi-stage RMSE loss, Adam with $152\,\mathrm{k}$3 and $152\,\mathrm{k}$4, and cosine annealing from $152\,\mathrm{k}$5 to $152\,\mathrm{k}$6 over 80 epochs (Cai et al., 28 Feb 2026).

Evaluation uses frame-wise averages of PSNR, SSIM, SAM, and ST-RRED, along with FLOPs and Params. In system-level comparison with PG-SVRT as the shared reconstructor, DD-CASSI achieved PSNR 41.52 dB, SSIM 0.9893, SAM 3.9084, and ST-RRED 23.25, outperforming PMVIS, SD-CASSI, and NDSSI (Cai et al., 28 Feb 2026). The interpretation given is that PMVIS and SD-CASSI suffer from severe spectral aliasing and do not preserve clear structural cues, whereas NDSSI preserves spatial fidelity and light throughput but its sparse spectral sampling limits spectral reconstruction capacity. DD-CASSI was therefore selected as the main architecture for subsequent experiments and for the real prototype.

Against image-based SCI baselines on DD-CASSI simulation, PG-SVRT achieved on the DynaSpec test set PSNR-D 41.82 dB, SSIM-D 0.9904, SAM-D 4.0118, and ST-RRED-D 27.14; on the KAIST test set it achieved PSNR-K 41.23 dB, SSIM-K 0.9882, SAM-K 3.805, and ST-RRED-K 19.35 (Cai et al., 28 Feb 2026). The model is reported to attain highest or tied highest PSNR on both KAIST and DynaSpec, best SAM, best ST-RRED, about 28.18 GFLOPs per frame, and 2.48M parameters. These results are used to support the claim that DynaSpec enables evaluation of temporal consistency and video-level spectral reconstruction, rather than only frame-wise fidelity.

4. Real DD-CASSI prototype and the imaging DynaSpec’s role

The DynaSpec imaging work extends beyond simulation to a real DD-CASSI prototype (Cai et al., 28 Feb 2026). The prototype uses the DD-CASSI architecture identified as optimal in simulation, including a dual-disperser structure and a binary mask trained in simulation, and captures real measurement sequences with spatial size $152\,\mathrm{k}$7.

Within this hardware setting, DynaSpec serves three roles. First, it provides training data: PG-SVRT and comparison methods are retrained on DynaSpec using the real mask pattern of the prototype in the forward simulation, aligning the network with the actual optical characteristics of the physical system. Second, it provides a transfer bridge: because DynaSpec consists of controlled, calibrated HSIs, simulation is treated as a reliable proxy for the real hardware, and the paper states that methods performing strongly on DynaSpec simulation also perform well on real prototype data. Third, it supports qualitative real-world evaluation: five real sequences are captured, pseudo-RGB images are generated from reconstructed HSIs, and PG-SVRT is reported to produce reconstructions with fewer artifacts, fewer distortions, and a more natural appearance, including a faithfully reconstructed Winnie-the-Pooh toy (Cai et al., 28 Feb 2026).

The dataset’s advantages are stated explicitly. It contains truly dynamic spectral content with physical object motion rather than only camera shifts; it offers 151 bands over 400–700 nm at 2 nm spacing; it provides $152\,\mathrm{k}$8 spatial resolution per frame; it is physically calibrated through spectral response correction, illumination removal, and intensity calibration; it is designed directly for reconstruction rather than downstream recognition; and because each frame is a full 3D HSI volume it can be used to simulate SD-CASSI, DD-CASSI, PMVIS, NDSSI, or new variants (Cai et al., 28 Feb 2026).

Its acknowledged limitations are also specific. The environment is controlled and mainly indoor, motions are manually designed, and the total of 30 sequences is moderate rather than massive. The paper nevertheless presents DynaSpec as a controlled benchmark and states that robust trends are observed under domain shift, including transfer from CAVE/KAIST plus DynaSpec training to real DD-CASSI scenes (Cai et al., 28 Feb 2026). A plausible implication is that the dataset is intended less as a natural-video corpus than as a calibrated instrument for inverse-problem research.

5. DynaSpec as context-aware dynamic speculative sampling

In large-vocabulary language modeling, DynaSpec is a context-aware dynamic shortlisting mechanism for speculative decoding (Zhang et al., 11 Oct 2025). Speculative decoding splits generation into a target model $152\,\mathrm{k}$9 and a smaller drafter $262\,\mathrm{k}$0. The drafter proposes a block of $262\,\mathrm{k}$1 tokens, the target verifies them once, and rejection-sampling-style verification preserves exactness. If $262\,\mathrm{k}$2 denotes expected per-token acceptance probability, the expected number of generated tokens per verification step is

$262\,\mathrm{k}$3

the average per-token latency is

$262\,\mathrm{k}$4

and the speedup over standard decoding is

$262\,\mathrm{k}$5

The paper’s motivation is that for a transformer with hidden width $262\,\mathrm{k}$6 and vocabulary size $262\,\mathrm{k}$7, the LM head incurs cost $262\,\mathrm{k}$8; for tiny drafters, this head can become a large fraction of total compute, so vocabulary growth directly slows draft-time inference (Zhang et al., 11 Oct 2025).

DynaSpec is presented as an alternative to static-shortlist methods such as FR-Spec and VocabTrim, which precompute a fixed sub-vocabulary $262\,\mathrm{k}$9 by ranking tokens according to corpus frequency and restricting the drafter to that subset (Zhang et al., 11 Oct 2025). The critique is twofold: frequency-ranked shortlists are corpus-dependent and require retuning to generalize across domains, and they suppress rare or domain-specific tokens, reducing the overlap between target and drafter distributions and thereby lowering acceptance.

The core idea is to make the shortlist context dependent. The vocabulary $\mathcal{O}(|V|d)$0 is partitioned into $\mathcal{O}(|V|d)$1 token clusters $\mathcal{O}(|V|d)$2, a lightweight meta-classifier scores these clusters for the current context, the top-$\mathcal{O}(|V|d)$3 clusters are selected, and the shortlist is the union

$\mathcal{O}(|V|d)$4

The drafter computes logits only over $\mathcal{O}(|V|d)$5, while the target model still verifies with the full vocabulary, so the overall speculative-decoding process remains exact (Zhang et al., 11 Oct 2025).

Token clustering is performed by spherical k-means on LM-head columns. For each token $\mathcal{O}(|V|d)$6, with LM-head column $\mathcal{O}(|V|d)$7, the normalized vector is

$\mathcal{O}(|V|d)$8

Spherical k-means over $\mathcal{O}(|V|d)$9 produces clusters $X \in \mathbb{R}^{T \times H \times W \times C}$0 and assignment map $X \in \mathbb{R}^{T \times H \times W \times C}$1. The clusters are not enforced to be balanced. The router $X \in \mathbb{R}^{T \times H \times W \times C}$2 is a small MLP that takes the embedding of the current token $X \in \mathbb{R}^{T \times H \times W \times C}$3 and the previous drafter hidden state $X \in \mathbb{R}^{T \times H \times W \times C}$4, concatenated into a vector in $X \in \mathbb{R}^{T \times H \times W \times C}$5, and outputs cluster scores

$X \in \mathbb{R}^{T \times H \times W \times C}$6

After sigmoid, these scores are treated as cluster-relevance probabilities (Zhang et al., 11 Oct 2025).

The router is trained offline using full-vocabulary EAGLE-2 traces on ShareGPT and UltraChat200K. For each step, the top-$X \in \mathbb{R}^{T \times H \times W \times C}$7 tokens from the full-vocabulary drafter define a cluster-level multi-label target through

$X \in \mathbb{R}^{T \times H \times W \times C}$8

and the router is trained with binary cross-entropy over clusters (Zhang et al., 11 Oct 2025). This procedure recasts shortlist selection as cluster-level multi-label classification.

6. Theory, systems design, and empirical behavior of the language-model DynaSpec

DynaSpec’s shortlist size is controlled by a position-aware cluster budget $X \in \mathbb{R}^{T \times H \times W \times C}$9. For the first two drafted positions, $1280 \times 1280$0 clusters are used; for later positions,

$1280 \times 1280$1

The rationale given is that early drafted tokens have disproportionate impact on acceptance and downstream tokens, so larger candidate sets should be allocated early and smaller ones later (Zhang et al., 11 Oct 2025). The paper notes that the same principle also improves FR-Spec when applied to a static frequency-ranked shortlist.

At the systems level, the router runs in parallel with the drafter core on separate CUDA streams. The per-step draft time is approximated by

$1280 \times 1280$2

where $1280 \times 1280$3 (Zhang et al., 11 Oct 2025). After router scoring and drafter hidden-state computation complete, a fused FUSED_INDEX_GEMM kernel gathers the selected LM-head columns and performs GEMM only on the shortlist. The resulting vocabulary-dependent term changes from $1280 \times 1280$4 to $1280 \times 1280$5, with $1280 \times 1280$6 small and $1280 \times 1280$7.

The theoretical analysis formalizes why dynamic supports can improve acceptance. If $1280 \times 1280$8 is the target distribution at context $1280 \times 1280$9 and the drafter is restricted to support $2\,\text{nm}$0, then the maximum possible acceptance over supports of size $2\,\text{nm}$1 is the target mass retained in that support: $2\,\text{nm}$2 The oracle dynamic $2\,\text{nm}$3-subset therefore maximizes retained target mass per context, and Theorem A states that the expected acceptance under such a context-dependent subset is at least that of any static subset of the same size, with strict inequality unless the static subset coincides almost surely with the context-specific top-$2\,\text{nm}$4 set (Zhang et al., 11 Oct 2025). DynaSpec is presented as an approximation to this dynamic oracle via clustering and routing.

Empirical results are reported on Llama-3-8B-Instruct with vocabulary size 128k, using an EAGLE-2 style drafter on a single NVIDIA A6000 GPU across seven tasks: machine translation, multi-turn conversation, RAG/QA, math, summarization, and code (Zhang et al., 11 Oct 2025). The primary metric is mean acceptance length. Full-vocabulary EAGLE-2 attains average mean acceptance 4.00. FR-Spec with a static 32k shortlist attains 3.74. DynaSpec attains 3.90 with average shortlist size 27,344, including task-wise average shortlist sizes such as 27,817 for machine translation, 27,020 for conversation, and 27,277 for code (Zhang et al., 11 Oct 2025). On code, the reported mean acceptance is 4.71 for DynaSpec versus 4.11 for FR-Spec; on machine translation it is 3.51 versus 3.38. The paper summarizes this as DynaSpec retaining about 97.5% of full-vocabulary EAGLE-2 mean acceptance while using a smaller average shortlist than FR-Spec.

The limitations identified for the language-model DynaSpec are distinct from those of the imaging dataset. It requires offline trace collection and router training; it depends on clustering quality; dynamic heads incur indexed-matmul overhead relative to dense GEMMs; and hyperparameters such as the number of clusters, router design, top-$2\,\text{nm}$5, $2\,\text{nm}$6, and the decay schedule may require tuning (Zhang et al., 11 Oct 2025). The text also notes that for narrow and stable deployment domains, a carefully tuned static shortlist with position-aware budgets may be simpler and sufficient.

Taken together, the two DynaSpec contributions exemplify a naming collision across unrelated research areas. One provides calibrated dynamic hyperspectral video ground truth for video-level spectral compressive imaging, paired with PG-SVRT and DD-CASSI benchmarking (Cai et al., 28 Feb 2026). The other provides a context-aware dynamic shortlist for large-vocabulary speculative decoding, combining token clustering, cluster routing, position-aware budgets, and fused indexed GEMM to mitigate the drafter’s $2\,\text{nm}$7 bottleneck (Zhang et al., 11 Oct 2025). The commonality lies only in the name; the technical substance, mathematical objects, and evaluation criteria are entirely different.