VEIL: Chart-Based TSC Diagnostic Framework

• VEIL is a framework that investigates how visual encoding hijacking affects time-series models by analyzing representation similarity, cross-encoding transfer, and attribution patterns.
• It employs linear CKA, PCA, UMAP, and cross-chart linear probing to quantify biases induced by different chart renderings.
• An attention-guided refinement inspired by HINT is used to diagnose and mitigate reliance on encoding-specific shortcuts in the classification process.

Searching arXiv for the specified VEIL paper and closely related work on chart-based time-series classification and HINT-style attention guidance. VEIL is a diagnostic framework for chart-based time-series classification (TSC) that examines how visual encoding choices shape learned representations in vision models trained on rendered time-series images. Rather than proposing a new classifier, it studies whether such models learn underlying temporal dynamics or exploit rendering-dependent cues such as stroke continuity, bar edges, point density, anti-aliasing patterns, axes, or whitespace. The framework centers on the phenomenon termed visual encoding hijacking: an encoding-dependent behavior in which learned representations align more with chart style than with temporal class structure. VEIL therefore treats chart-based TSC as a representation and measurement problem, not merely a modeling convenience (Sooraj et al., 6 Jul 2026).

1. Problem formulation and conceptual distinction

Chart-based TSC converts a univariate time series into a 2D image and then applies a vision backbone such as a CNN or ViT. In the setting studied by VEIL, the sequence is first rendered as a line chart, area chart, bar chart, scatter plot, or a transform-derived image such as a Gramian Angular Field or Markov Transition Field, and then classified with an image model. This paradigm is attractive because it reuses mature vision architectures, including pretrained ones, and permits saliency overlays such as Grad-CAM on visually interpretable charts (Sooraj et al., 6 Jul 2026).

The central problem is that charts preserve neither only nor purely temporal structure. They also inject encoding-specific regularities that are not inherent properties of the original sequence. A line chart adds continuity and slope cues; an area chart adds filled regions and region boundaries; a bar chart adds rectilinear edges and vertical boundaries; a scatter plot adds point density and local clustering. A high-performing classifier may therefore succeed by internalizing properties of the rendering rather than properties of the time series itself.

VEIL distinguishes visual encoding hijacking from legitimate encoding-specific evidence. The distinction is important. Some encoding-specific cues can honestly reflect the signal, such as line continuity or bar height. Hijacking, by contrast, refers to cases in which the model’s internal organization and decision process are dominated by the rendering style. VEIL argues that this cannot be diagnosed from accuracy alone, or from a single auxiliary metric. Instead, it requires joint evidence from representation similarity, cross-encoding transfer, and attributional behavior.

2. Framework architecture and diagnostic machinery

VEIL is organized as a four-stage pipeline: rendering, learning, diagnostics, and attention-guided refinement. Multiple encodings $$e \in \{\text{line}, \text{area}, \text{bar}, \text{scatter}\}$$ are generated from the same underlying time-series dataset, separate CNN encoders $f_e$ are trained, and the resulting feature spaces and predictions are compared across encodings (Sooraj et al., 6 Jul 2026).

The principal representation-level diagnostic is linear CKA between penultimate-layer activation matrices $X$ and $Y$: $$\mathrm{CKA}(X, Y) = \frac{\|Y^\top X\|_F^2}{\|X^\top X\|_F \, \|Y^\top Y\|_F}.$$ High CKA indicates aligned representations across encodings; low CKA indicates that different chart types induce materially different feature spaces. VEIL supplements this with geometric diagnostics. PCA is applied to each encoding’s features, and the smallest number of principal components explaining $90\%$ of the variance is recorded as a coarse proxy for representation compactness. UMAP provides two-dimensional projections of embeddings, permitting direct inspection of whether samples cluster primarily by class or by chart type.

The second major diagnostic is cross-chart linear probing. An encoder $f_{e_{\text{train}}}$ trained on one encoding is frozen, its penultimate-layer features are extracted for samples rendered in another encoding $e_{\text{probe}}$, and a logistic regression classifier is trained on those frozen features. High off-diagonal accuracy implies that the learned representation contains task-relevant information that transfers across encodings; low off-diagonal accuracy indicates encoding-specific specialization.

The third diagnostic is attribution analysis via Grad-CAM on the final convolutional layer. Heatmaps are computed for correctly and incorrectly classified examples, then qualitatively categorized according to whether attention falls on signal structure, encoding-specific features, or irrelevant elements such as axes, gridlines, and whitespace. VEIL complements attribution with encoding-targeted perturbations: Gaussian blur for line and area charts, bar merging for bar charts, and alpha fading for scatter plots. These are used to test how strongly predictions depend on specific visual elements.

To summarize cross-encoding dependence at the dataset level, VEIL defines Encoding Sensitivity Indices. For CKA, the index is $1 - \text{mean off-diagonal CKA}$; for linear probes, it is $1 - \text{mean off-diagonal probe accuracy}$. High ESI values indicate strong dependence on encoding choice.

3. Experimental basis: datasets, encodings, and training protocol

VEIL is evaluated on 31 datasets from the UCR Time Series Classification Archive, spanning binary and multi-class problems, short to long sequences, and domains including ECG5000, FordB, Wafer, ArrowHead, GunPoint, Adiac, SharePriceIncrease, Yoga, Crop, Strawberry, and RefrigerationDevices (Sooraj et al., 6 Jul 2026). The study emphasizes that sequence length alone does not explain encoding sensitivity; class structure is more predictive.

All series are rendered with matplotlib at a fixed resolution of $f_e$0 pixels. The principal chart encodings are line, area, bar, and scatter. Other visual choices such as colors and axes are kept consistent to isolate the effect of encoding type. As a robustness check beyond standard charts, the same protocol is later applied to five non-chart transforms: GASF, MTF, Recurrence Plot, Continuous Wavelet Transform scalograms, and STFT spectrograms.

The learning setup uses a single CNN backbone across all encodings. Convolution, nonlinearity, and pooling layers feed a penultimate feature layer used for CKA, PCA, UMAP, and linear probes, while the final convolutional layer is reserved for Grad-CAM. The final layer is a softmax classifier. Using a shared architecture across encodings is methodologically central: it attributes differences in learned structure to the visual encoding rather than to architectural variation.

Training is standardized across datasets. The optimizer is Adam with learning rate $f_e$1 and weight decay $f_e$2, early stopping uses patience 10, and there is no per-dataset hyperparameter tuning. The baseline objective is standard cross-entropy,

$f_e$3

with no additional regularizers. This controlled protocol is intended to expose encoding effects directly.

4. Empirical characterization of visual encoding hijacking

VEIL reports a strongly heterogeneous empirical landscape. Some datasets are effectively encoding-invariant. In these cases, cross-encoding CKA remains high, approximately $f_e$4 to $f_e$5, and transferability across chart types remains strong. Examples include Wine and other simple binary datasets. Here, models appear to capture class structure robustly across renderings (Sooraj et al., 6 Jul 2026).

Other datasets are clearly encoding-divergent. For FordB, SharePriceIncrease, RefrigerationDevices, and Adiac, cross-chart CKA falls to approximately $f_e$6 to $f_e$7, indicating that the same underlying time series, when rendered differently, induce substantially different representation manifolds. Intermediate cases such as GunPoint and Yoga show partial alignment: area-bar pairs align relatively well, around $f_e$8 to $f_e$9, whereas line and scatter diverge more strongly, around $X$0 to $X$1.

The dataset-level sensitivity indices reinforce this picture. $X$2 peaks at 0.524 for SharePriceIncrease, 0.464 for FordB, and 0.49 for RefrigerationDevices. $X$3 peaks at 0.883 for Adiac and 0.806 for Beef. Binary datasets such as Wafer and Strawberry can remain insensitive even with large sequence length, whereas multi-class datasets such as Adiac can be highly sensitive even when sequences are shorter. This suggests that class count and class complexity correlate more strongly with encoding sensitivity than sequence length.

Cross-chart probes mirror the CKA results. ECG5000 exhibits off-diagonal probe accuracies between 0.64 and 0.94, close to diagonal values of 0.99 to 1.00, showing that features learned under one encoding transfer well to others. Yoga, by contrast, has diagonal probe accuracies near 0.99 to 1.00 but off-diagonals of only 0.55 to 0.76, indicating substantial specialization to chart type. UMAP projections support the same interpretation: for datasets such as Yoga and FordB, embeddings cluster primarily by encoding rather than by class, whereas ECG5000 shows more mixing of class and chart structure.

Attribution analysis adds a third, qualitatively different axis of evidence. Grad-CAM overlays frequently highlight encoding-specific artifacts—bar boundaries, dense scatter clusters, filled area boundaries—rather than only temporally meaningful regions. Incorrect predictions often correspond to saliency on irrelevant elements such as axes, borders, or whitespace. VEIL therefore treats attribution shifts across encodings as a necessary complement to similarity and transfer diagnostics, rather than as a stand-alone explanation.

Perturbation studies show that degrading encoding-specific visual elements often reduces accuracy. Bar charts are comparatively robust, with an average drop around $X$4, suggesting that they rely more on coarse structure than on fine visual detail. VEIL explicitly cautions that perturbation sensitivity alone does not prove shortcut use, because some encoding-specific evidence may still be semantically legitimate. The claim of hijacking rests on the conjunction of low cross-encoding alignment, weak transfer, and attributional drift.

5. Attention-guided refinement as diagnostic intervention

VEIL includes a mitigation experiment inspired by HINT, adapted to chart-based TSC in a self-supervised form. The procedure is iterative. A baseline CNN is first trained on chart images. Grad-CAM is then computed per input, the most salient region is masked out with a white bounding box, and the masked image is passed back through the model to obtain a second Grad-CAM map. That second map serves as an alignment target for a new training run, in which the loss becomes

$X$5

Here $X$6 penalizes divergence between the current Grad-CAM map and the target attention pattern, while $X$7 weights the alignment term (Sooraj et al., 6 Jul 2026).

The empirical behavior is selective rather than uniform. On encoding-sensitive datasets, attention guidance can yield large gains. Reported examples include ToeSegmentation2, $X$8 (+38.46%); FacesUCR, $X$9 (+20.54%); Crop, $Y$0 (+20.92%); Ham, $Y$1 (+23.81%); Adiac (+16.62%); the Cricket datasets (+8–11%); and FordB (+10.62%). These gains occur precisely where VEIL’s other diagnostics also indicate strong encoding sensitivity: low cross-encoding CKA, high $Y$2, and chart-type clustering in UMAP.

On encoding-invariant datasets, however, the effect is minimal or null. ECG5000 changes from 93.44 to 93.29, Wafer from 99.50 to 99.59, and datasets such as Wine, Beef, BeetleFly, and Lightning2 show no change. More importantly, attention guidance can be harmful. Computers drops from 69.60 to 33.60, SonyAIBORobotSurface1 from 57.07 to 42.93, and PhalangesOutlinesCorrect by 3.85%.

VEIL therefore does not present attention guidance as a general remedy. Its more precise conclusion is that HINT-style refinement functions primarily as a diagnostic intervention. When multiple diagnostics agree that representations are encoding-sensitive, redirecting attention can help and may reduce encoding dependence. When those signals are absent, or when the model-derived target saliency is itself misleading, the method offers limited or negative benefit.

6. Broader significance, limitations, and research directions

VEIL situates chart-based TSC within the broader tradition of graphical perception, but transfers the question from human readers to vision models. Classical graphical perception asks which encodings best support human estimation and trend detection. VEIL asks how machines perceive charts, and shows that models can be influenced by chart design in ways that do not simply mirror human reasoning (Sooraj et al., 6 Jul 2026).

Several implications follow. First, chart encoding acts as an inductive bias in its own right. A line chart foregrounds continuity; a scatter plot foregrounds local point patterns; a bar chart foregrounds vertical boundaries and rectilinear geometry. Second, accuracy on a single encoding is not sufficient evidence of robust temporal understanding. A model that performs well on line charts may fail on scatter charts for the same series, revealing dependence on rendering-specific cues rather than encoding-invariant signal structure. Third, VEIL argues for diagnostic evaluation protocols that combine CKA, cross-encoding probes, attribution analysis, and perturbation testing, rather than reporting only per-encoding accuracy.

The paper also makes the associated controversies explicit. VEIL does not claim that all encoding-specific evidence is spurious. It acknowledges that some such cues may faithfully reflect the underlying series. It also avoids a stronger causal claim that it cannot support: the framework shows reliance on encoding-specific cues, but not always that those cues are task-irrelevant shortcuts. This is why the paper frames chart-based TSC as a measurement problem rather than a settled indictment of image-based time-series modeling.

The stated limitations are equally specific. VEIL studies CNNs trained from scratch, not transformers, large pretrained vision models, or multimodal systems. It investigates four chart types plus several transforms, but not the full space of encodings such as multi-channel charts, color encodings, or non-standard glyphs. Its attribution analysis relies heavily on Grad-CAM, which is itself imperfect. The HINT adaptation is self-supervised rather than annotation-based, so it may propagate baseline biases. And the empirical basis is UCR-style univariate TSC; multivariate, long-horizon, or highly irregular time series may behave differently.

The proposed research directions follow directly from these limits: encoding-robust models that explicitly seek encoding-invariant representations; encoding-aware benchmarks requiring consistency across chart types; more principled visualization designs for machine consumption; and alternative diagnostics, including causal testing, to separate useful encoding-specific evidence from harmful shortcut exploitation more rigorously. In that sense, VEIL reframes visualization choice in chart-based TSC from a preprocessing detail into a substantive component of the learning problem itself.