Misviz: Detecting Faulty Visualizations

• Misviz is a research field focused on identifying and correcting visualizations that mislead through design flaws or poor data fidelity.
• It employs taxonomies and benchmarks, such as Misviz and Misviz-synth, to evaluate methods ranging from rule-based linters to zero-shot multimodal models.
• The research highlights actionable metrics and detailed criteria to improve chart clarity, mitigate misinformation, and ensure robust visualization design.

Misviz denotes a cluster of research problems centered on misleading, low-quality, or otherwise unreliable visual communication, and on the computational methods used to detect, explain, test, and repair such failures. In recent work, the term spans broad quality frameworks for static charts, conceptual models of “visualization mirages,” benchmark datasets for misleader detection, and interactive systems that localize and correct issues in chart images (Sawicki et al., 2022, McNutt et al., 2020, Tonglet et al., 29 Aug 2025, Das et al., 6 Aug 2025). Across these works, misviz is not restricted to deliberate deception: it also includes accidental bad design, weak evidentiary grounding, unreadability, confusion, and silent failures that make a chart appear plausible while weakening or invalidating its message.

1. Conceptual scope and problem formulation

In visualization-quality research, misviz is framed as a broad class of failures in which visualizations are misleading, unreadable, confusing, overly complex, or otherwise incapable of conveying the intended message clearly and faithfully. This framing explicitly connects low-quality charts to misinformation, because such graphics can “spread misinformation” and “fail to deliver their message to the viewer” (Sawicki et al., 2022). The emphasis is therefore not only on deception, but also on design or data choices that distort understanding, obscure evidence, or make interpretation unnecessarily difficult.

A closely related concept is the visualization mirage, defined as a visualization where a cursory reading appears to support a particular message from the data, but closer re-examination of the visualization, backing data, or analytical process would invalidate or significantly weaken that support (McNutt et al., 2020). This definition highlights a characteristic failure mode of misviz: the chart need not look broken. On the contrary, it may appear credible and analytically useful until a deeper inspection reveals fragility, bias, or invalid inference.

The mirage formulation situates misviz within a full visual analytics pipeline. The relevant stages are curating, wrangling, visualizing, and reading, and failures may arise at any stage. Missing or repeated records, inconsistent categories, outliers, cherry picking, confusing imputation, truncated axes, dual axes, concealed uncertainty, and reader-side assumptions of causality are all identified as sources of misleading conclusions (McNutt et al., 2020). This suggests that misviz is not solely a property of the rendered graphic; it is often a mismatch between data, design, and interpretation.

A common misconception is that misviz refers only to obviously deceptive charts such as truncated bars or 3D pie charts. The literature is broader. It includes misleadingness, unreadability / incomprehensibility, confusion / ambiguity, and poor data fidelity or weak evidentiary grounding, including the rejection of “guesstimates” in favor of real data (Sawicki et al., 2022). It also includes task-sensitive failures, where a chart may be harmless for one analytical purpose and misleading for another (McNutt et al., 2020).

2. Taxonomies, principles, and evaluative codices

One major line of work treats misviz as the failure to satisfy principled criteria for correct visualization. VisQualdex defines four “pillars” of a correct visualization: real data instead of guesstimates, clarity and readability instead of incomprehensibility and ambiguity, simplicity and summarization instead of complexity and raw data, and guidance and objectivity instead of manipulation and subjectivity (Sawicki et al., 2022). In this formulation, misviz is the violation of one or more of these principles.

VisQualdex operationalizes this view through a taxonomy of seven categories: Subjective, Theme, Coordinates, Geometry, Guides, Perception, and Data (Sawicki et al., 2022). The categories are described as complete and disjoint: there is no “other” category, and each question belongs to exactly one category. This structure is intended to address perceived limitations of prior checklist-style evaluation methods, including lack of hierarchy, mixing of very general and very specific advice, non-universal rules, oversimplified scoring, and incomplete coverage.

The framework contains 60 criteria, each phrased as a yes/no question. A negative answer triggers only when something is incorrect; some questions may be inapplicable; a single bad practice can trigger multiple questions; and the impact of a negative answer may vary greatly (Sawicki et al., 2022). A central methodological point is that visualization quality cannot be judged solely by the fraction of positive answers. One severe flaw may make a visualization fundamentally wrong even when most criteria are satisfied.

Examples of criteria illuminate how misviz is decomposed into concrete design failures. “Is there not too many colors representing the data?” targets perceptual overload and poorly organized gradients. “Does it omit or utilize properly the third dimension?” addresses distortions arising from illusory depth, angle misjudgment, hidden data points, and projection artifacts. “Does the visualization obey the reading gravity?” formalizes narrative and perceptual ordering relative to culturally expected reading patterns (Sawicki et al., 2022). These criteria treat misviz as a combination of perceptual, textual, geometric, and data-integrity defects rather than a single undifferentiated problem.

A broader taxonomy also appears in later system work. MisVisFix builds on Lo et al.’s taxonomy of 74 distinct misleading visualization issues, including truncated axes, dual y-axes, 3D effects, inappropriate encodings, misleading color choices, selective data presentation, misrepresentation of findings, cherry-picking, and misleading framing (Das et al., 6 Aug 2025). The Misviz benchmark narrows this broader space to 12 categories chosen because they appeared at least three times in the source corpus, were genuinely deceptive rather than merely confusing, and were detectable without domain-specific expertise (Tonglet et al., 29 Aug 2025).

3. Benchmarks and annotation protocols

The benchmark literature turns misviz into a multi-label detection problem: determine whether a chart is misleading at all and, if so, identify which specific misleaders are present. The Misviz benchmark contains 2,604 real-world visualizations annotated with 12 misleader types, while its synthetic companion, Misviz-synth, is designed to support model training and controlled evaluation (Tonglet et al., 29 Aug 2025).

Resource	Composition	Intended use
Misviz	2,604 real-world visualizations; 12 misleader types	Detection of misleading visualizations in the wild
Misviz-synth	Intended benchmark size 81,814 synthetic visualizations	Training and controlled evaluation
Axis metadata in Misviz-synth	Seq, Axis, Label, Relative position	Axis extraction, rule-based linting, axis-aware models

Misviz is explicitly multi-label. 70% of the visualizations contain up to three misleaders, and the rest are non-misleading. The split is 5% development, 15% validation, and 80% held-out test (Tonglet et al., 29 Aug 2025). The 12 categories are: Misrepresentation, 3D, Truncated axis, Inappropriate use of pie chart, Inconsistent tick intervals, Dual axis, Inconsistent binning size, Discretized continuous variable, Inappropriate use of line chart, Inappropriate item order, Inverted axis, and Inappropriate axis range.

The real-world corpus is assembled from three sources: 353 misleading charts from the corpus of Lo et al. (2022), 964 charts from the “WTF Visualizations” website, and 474 charts from Reddit communities, especially r/dataisugly for misleading examples and r/dataisbeautiful for non-misleading ones (Tonglet et al., 29 Aug 2025). Duplicates were removed using perceptual hashing, and low-resolution images, examples that explicitly named the misleader, and charts showing misleading and corrected versions side by side were excluded.

Annotation was performed in three Prolific tasks, each with three annotators paid £9/hour and required to hold a PhD. The three tasks were chart-type annotation, misleader annotation, and non-misleading verification, with agreement on an overlapping set of 30 items measured by Fleiss’ $\kappa$. Reported inter-annotator agreement was 0.71 for chart type, 0.53 for misleader annotation, and 0.78 for non-misleading verification (Tonglet et al., 29 Aug 2025). These figures indicate that misleader identification is materially harder than basic chart-type assignment or confirmation that a chart is not misleading.

Misviz-synth differs in purpose and construction. It contains synthetic visualizations generated from real-world tables collected from Wikipedia via TabFact and from Our World in Data. Generation is two-stage: first, tables are prepared using a type system over columns; second, Matplotlib plotting functions render charts and inject one applicable misleader per chart (Tonglet et al., 29 Aug 2025). The dataset is stratified into 93% / 2% / 5% train/validation/test splits, with bar 46.1%, line 46.7%, and pie 7.2%; 78% of instances are misleading; there is one chart type per instance and at most one misleader per chart. Because of Matplotlib limitations, Misviz-synth does not include maps and therefore omits the map-specific misleader discretized continuous variable (Tonglet et al., 29 Aug 2025).

A notable design feature of Misviz-synth is the inclusion of axis metadata. For bar and line charts, the metadata schema is a four-column table: Seq, Axis, Label, and Relative position (Tonglet et al., 29 Aug 2025). This additional structure makes the dataset suitable not only for end-to-end image classification, but also for chart de-rendering, axis extraction, and rule-based linting.

4. Detection models, linting, and interactive correction

The Misviz benchmark evaluates three main families of methods: zero-shot multimodal LLMs (MLLMs), a rule-based linter, and fine-tuned classifiers (Tonglet et al., 29 Aug 2025). The zero-shot MLLMs include GPT-4.1, GPT-o3, Gemini-1.5-Flash, Gemini-1.5-Pro, Qwen2.5-VL-7B / 32B / 72B, and InternVL3-8B / 38B / 78B. These models receive the chart image plus definitions of the 12 misleaders and are asked to return up to three labels or “no misleader.”

The rule-based linter operates over axis metadata rather than underlying data tables. It can detect truncated axis, inverted axis, dual axis, inconsistent tick intervals, inconsistent binning size, and inappropriate item order (Tonglet et al., 29 Aug 2025). This choice of representation increases coverage relative to purely table-based rules, but it also leaves the linter unable to handle visually or contextually subtle cases such as misrepresentation or inappropriate axis range.

The fine-tuned classifiers include an image-only classifier and an image + axis metadata classifier. The image encoder is based on TinyChart; axis metadata is encoded with TaPas; and the combined representation is passed through an MLP classifier (Tonglet et al., 29 Aug 2025). Because Misviz itself does not include axis metadata, the benchmark also fine-tunes DePlot on Misviz-synth to predict axis metadata from images, creating an intermediate extraction step for linter- and axis-aware methods.

Evaluation uses Accuracy, Precision for the misleading class, Recall for the misleading class, Macro-F1, Exact Match (EM), and Partial Match (PM). The benchmark defines EM = 1 iff predicted misleaders exactly equal the gold set, and PM = 1 iff the predicted set is a subset of the gold set (Tonglet et al., 29 Aug 2025). These set-based criteria are important because much of the difficulty lies not in deciding that a chart is suspicious, but in identifying the exact misleader set.

On Misviz, the strongest zero-shot results are reported for GPT models. GPT-o3 achieves Acc 83.5, Pre 86.7, Rec 90.1, F1 80.0, EM 58.9, PM 67.5, and GPT-4.1 achieves Acc 84.1, Pre 84.5, Rec 94.3, F1 79.6, EM 53.6, PM 64.0 (Tonglet et al., 29 Aug 2025). On the bar/line/pie comparison subset, GPT-o3 remains best among zero-shot MLLMs on Misviz with Acc 84.9, Pre 88.8, Rec 90.4, F1 80.9, EM 59.4, PM 67.9. On Misviz-synth, the strongest synthetic results include F1 75.5, EM 67.4 for the linter with DePlot-predicted axes, F1 71.7, EM 80.4 for the image + axis classifier, and F1 66.0, EM 76.8 for the image-only classifier (Tonglet et al., 29 Aug 2025). The main pattern is that real-world charts are hard for all models, while synthetic charts are easier for rule-based and trained methods, especially when axis metadata is available.

MisVisFix extends detection into explanation, localization, and repair. It is an interactive dashboard that uses Claude 3.7 and GPT-4.5 to support the full workflow of detecting, explaining, localizing, and correcting misleading visualizations from bitmap chart images (Das et al., 6 Aug 2025). The pipeline begins with preprocessing, then uses Claude 3.7 for data extraction and chart-type detection, runs issue detection against the full 74-issue taxonomy, classifies issues as Major, Minor, or Potential concerns, localizes them using percentage-based coordinates such as top_gap and left_gap, and generates corrected charts via Python code using Matplotlib, Seaborn, and other visualization libraries. A chat interface supports iterative modification, and a learning mechanism allows users to add newly observed issues to the system’s knowledge base.

The reported evaluation dataset contains 450 visualizations, of which 360 are misleading and 90 are valid. MisVisFix reports Precision 0.94, Recall 0.98, and F1 0.96, compared with F1 = 0.69 for an LLM-only baseline and F1 = 0.61 for VizLinter (Das et al., 6 Aug 2025). By chart type, F1 is 0.97 for bar charts, 0.96 for line charts, 0.93 for pie/donut charts, 0.92 for scatterplots, and 0.91 for other types. Severity classification accuracy is 87.5%, with 94.2% for major issues, 85.6% for minor issues, and 82.8% for potential issues; localization precision is 91.3% average precision, with 96.4% for axes and 94.8% for textual components (Das et al., 6 Aug 2025). Structural issue detection is stronger for GPT-4.5, while Claude 3.7 is stronger for contextual issue detection, supporting the system’s dual-model design.

5. Visualization mirages and metamorphic testing

A distinct methodological tradition does not classify misleaders directly. Instead, it attempts to surface charts whose apparent messages are fragile under controlled perturbations. In this literature, a visualization mirage is a silent failure of visual communication: the apparent conclusion is not robust, even though the chart looks plausible (McNutt et al., 2020).

The technical response is metamorphic testing for visualization (MTV), adapted from software testing. Rather than asking whether a chart is correct in an oracle-based sense, MTV asks whether a predictable change to the input produces the expected change in the output. The formal definition given is

$$MTV:: (Eq, \alpha, \omega) \Rightarrow (spec, data) \Rightarrow Boolean$$

and

$$MTV(Eq, \alpha, \omega)x = Eq(v(\alpha(x)), \omega(v(x))).$$

Here, $x = (spec, data)$, $Eq$ is an equality or similarity check on outputs, $\alpha$ is an input perturbation, $\omega$ is the expected transformation on the output, and $v$ is the visualization process (McNutt et al., 2020).

The framework is linked to Algebraic Visualization Design (AVD) through the commutativity relation

$$v \circ r_2 \circ \alpha = \omega \circ v \circ r_1,$$

with the interpretation that trivial data changes should lead to trivial visual changes, and important data changes should lead to correspondingly important visual changes (McNutt et al., 2020). The paper characterizes failures as hallucinators, confusers, and non-sequiturs.

The proof-of-concept system operates on static Vega-Lite charts and focuses mainly on cases where $\omega$ is the identity. Four tests are described. Shuffle permutes row order and checks whether the image stays the same, targeting overplotting and order-dependent rendering effects. Bootstrap resamples rows within each bar and checks whether the qualitative ranking of bar heights remains stable, targeting fragile summaries, dependence on outliers, and instability due to small sample size. Contract Records equalizes sample sizes by sampling other groups down to the minimum group size, targeting differences driven by unequal numbers of records. Randomize permutes the assignment between values and categories, probing whether an apparently structured signal survives null-like perturbation (McNutt et al., 2020).

The evaluation uses a simulation study with synthetic two-group Gaussian datasets varying mean difference, sample size, outliers, and variance. The setup produces 30 datasets per condition, 4 manipulations, 5 effect sizes, for a total of 600 datasets (McNutt et al., 2020). Reported findings include that more severe manipulations generally produced stronger instability signals, the bootstrap test was especially good for outliers and variance, randomize was useful for detecting true group differences, and contract records was useful for sample-size disparities. In contrast to benchmark-based multi-label detection, MTV does not primarily ask which named misleader is present; it asks whether the chart’s message is robust to benign metamorphoses. This suggests a complementary relationship between taxonomy-driven detection and perturbation-driven verification.

6. Adjacent usages and disambiguation

The proximity of the labels Misviz and MultiViz invites confusion, but the two refer to different research problems. MultiViz is a visualization and interpretability framework for multimodal models, not a misleading-chart detector (Liang et al., 2022). Its motivation is that multimodal model behavior is too complex to explain with a single explanation view, so it scaffolds interpretability into four complementary stages: Unimodal importance, Cross-modal interactions, Multimodal representations, and Multimodal prediction. The framework is designed to operate across multiple modalities, tasks, and models, and experiments on 8 trained models across 6 real-world tasks show that the stages together enable users to simulate model predictions, assign interpretable concepts to features, perform error analysis on model misclassifications, and debug models (Liang et al., 2022).

MultiViz’s formal setup assumes a multimodal dataset

$$MTV:: (Eq, \alpha, \omega) \Rightarrow (spec, data) \Rightarrow Boolean$$0

with a trained model

$$MTV:: (Eq, \alpha, \omega) \Rightarrow (spec, data) \Rightarrow Boolean$$1

and defines separate modules for unimodal attribution and cross-modal interaction analysis (Liang et al., 2022). Its empirical studies include a user simulation experiment with 21 human users, 35 total hours of interaction, and seven ablation settings; on VQA 2.0, full MultiViz achieves 81.7 ± 2.9 correctness and 0.86 agreement, while on CMU-MOSEI it achieves 100.0 ± 0.0 correctness and 1.00 agreement. The framework also supports debugging: in a case study on LXMERT fine-tuned on VQA 2.0, inspection of the top 5 penultimate-layer neurons on erroneous examples revealed that 2 of the 5 neurons were consistently related to color questions, especially blue; the model had only 5.5% accuracy on blue-related points, accounting for 8.8% of all errors, and the failure was traced to a preprocessing bug in which Faster R-CNN image preprocessing had swapped RGB to BGR (Liang et al., 2022). This is interpretability research rather than misviz detection, but it shares a concern with visualization as a diagnostic interface for complex behavior.

A second adjacent usage appears in multimodal misinformation detection for short videos. VMID is a framework that converts video frames, OCR/subtitles, audio transcription, and metadata into a unified textual prompt and feeds that prompt into a LoRA-tuned LLM for classification as misinformation, debunking, or real (Zhong et al., 2024). The system uses CogVLM2 for keyframes, VSE for subtitle extraction, Whisper for audio, and metadata such as upload time, likes, comments, and author information. Reported results include 90.93% accuracy for VMID (Qwen2.5) versus 81.05% for SV-FEND, with a best variant using GLM4 reaching Accuracy 91.20%, F1 91.15%, Precision 91.12%, and Recall 91.22% (Zhong et al., 2024). A plausible implication is that misviz research increasingly intersects with broader multimodal misinformation analysis: chart images, short videos, and other media are being treated as heterogeneous evidence objects requiring detection, reasoning, and repair.

Taken together, these literatures position misviz as both a visualization-specific problem and part of a wider trustworthiness agenda. The visualization-specific strand focuses on chart design, perceptual distortion, data fidelity, and multi-label misleader identification. The broader strand extends similar concerns to interpretability pipelines and multimodal misinformation systems. What unifies them is the technical question of how to expose when visual or multimodal artifacts support conclusions that they should not support, or support them more strongly than the evidence warrants.