Critical Design Strategy (CDS)

• Critical Design Strategy (CDS) is a three-stage heuristic evaluation method that guides designers to reflect on and improve visualization artefacts.
• It combines qualitative first-impression elicitation with quantitative scoring across multiple perspectives, such as User and Visual Marks.
• CDS provides actionable next steps and iterative refinement, making it especially useful for novice visualization designers and academic settings.

Searching arXiv for the specified paper and closely related background. Critical Design Strategy (CDS) is a structured method for heuristically evaluating visualisation designs through reflection and critical thought. It is presented as a three-stage process for examining a visualisation artefact, with the explicit aim of helping designers think critically and make informed improvements using heuristic evaluation. The method is positioned as particularly useful when developing a visual tool or pioneering a novel visualisation approach, where identifying areas for enhancement can be difficult, and it is described as especially relevant to visualisation designers and tool developers who are new to the field, including students studying visualisation in higher education (Roberts et al., 7 Aug 2025).

1. Definition, scope, and theoretical basis

CDS is defined both informally and formally. Informally, it is a three-step heuristic evaluation method that helps a designer, termed the “appraiser,” reflect on a visualization artefact. Formally, it operates on a set of artefacts $\mathcal{A}$ through three stage functions:

$$S_1:\mathcal{A}\to P_1,\qquad S_2:\mathcal{A}\times P_1\to P_2,\qquad S_3:\mathcal{A}\times P_1\times P_2\to R$$

with

$$P_1=\{(Name, Summary, W_5)\mid Name\in Strings,\ Summary\in Strings,\ W_5\subseteq W,\ |W_5|=5\}$$

$$P_2=\{Score(p,q)\in\{-2,-1,0,1,2\}\mid p\in P,\ q\in Q_p\}$$

where $$P=\{\text{User, Environment, Interface, Components, Design, Visual Marks}\}$$ and $Q_p$ is the set of five heuristics for perspective $p$. The overall pipeline is written as

$CDS(A)=S_3\big(A,S_1(A),S_2(A,S_1(A))\big)\in R$

where $R$ is a report or set of reflections plus next-step recommendations (Roberts et al., 7 Aug 2025).

Theoretical foundations are explicitly identified. These include rhetoric and critical thinking, citing Aristotle, Cicero, and Facione 1990; heuristic evaluation, citing Nielsen 1990, Zuk and Carpendale 2006, and Forsell and Johansson 2010; information-visualisation frameworks, citing Bertin 1983, Shneiderman 1996, and Munzner 2009; and explanatory frameworks in education, citing Treagust and Harrison 2000 (Roberts et al., 7 Aug 2025). This situates CDS at the intersection of evaluative method, visualisation theory, and pedagogy.

A plausible implication is that CDS is intended less as a narrowly instrumental checklist than as a reflective framework that binds critical vocabulary, structured judgement, and design iteration into a single evaluative routine.

2. Three-stage procedure

The method is organised into three stages: Overview, Detail, and Review. Each stage has a distinct function and output (Roberts et al., 7 Aug 2025).

In Stage 1, denoted $S_1(A)$, the appraiser captures the essence of the artefact by assigning a 2–3 word title, writing a 1–2 sentence summary, and choosing five adjectives from a fixed set of twenty. The output is $$S_1:\mathcal{A}\to P_1,\qquad S_2:\mathcal{A}\times P_1\to P_2,\qquad S_3:\mathcal{A}\times P_1\times P_2\to R$$0. This stage is designed to form initial impressions of the design without immediately collapsing evaluation into detailed scoring.

In Stage 2, denoted $$S_1:\mathcal{A}\to P_1,\qquad S_2:\mathcal{A}\times P_1\to P_2,\qquad S_3:\mathcal{A}\times P_1\times P_2\to R$$1, the appraiser performs an in-depth critique using six perspectives, each containing five heuristic questions. Every question receives a score $$S_1:\mathcal{A}\to P_1,\qquad S_2:\mathcal{A}\times P_1\to P_2,\qquad S_3:\mathcal{A}\times P_1\times P_2\to R$$2. Summary scores are then computed as

$$S_1:\mathcal{A}\to P_1,\qquad S_2:\mathcal{A}\times P_1\to P_2,\qquad S_3:\mathcal{A}\times P_1\times P_2\to R$$3

The paper describes this as a 5-point Likert scale from $$S_1:\mathcal{A}\to P_1,\qquad S_2:\mathcal{A}\times P_1\to P_2,\qquad S_3:\mathcal{A}\times P_1\times P_2\to R$$4 to $$S_1:\mathcal{A}\to P_1,\qquad S_2:\mathcal{A}\times P_1\to P_2,\qquad S_3:\mathcal{A}\times P_1\times P_2\to R$$5, anchored as Poor $$S_1:\mathcal{A}\to P_1,\qquad S_2:\mathcal{A}\times P_1\to P_2,\qquad S_3:\mathcal{A}\times P_1\times P_2\to R$$6 Good, with perspective-specific semantic differentials for each heuristic in later versions (Roberts et al., 7 Aug 2025).

In Stage 3, denoted $$S_1:\mathcal{A}\to P_1,\qquad S_2:\mathcal{A}\times P_1\to P_2,\qquad S_3:\mathcal{A}\times P_1\times P_2\to R$$7, the appraiser reflects on the outputs of Stages 1 and 2, interprets both overall and per-perspective scores, and writes next steps, such as redesigning layout, refining marks, or improving labels. The result is a narrative summarising strengths, weaknesses, and action items (Roberts et al., 7 Aug 2025).

The procedure therefore combines qualitative and quantitative elements. The paper’s own summary states that CDS guides designers through Overview, Detail, and Review to deepen critical reflection, produce quantitative-qualitative scores, and drive concrete improvements in visualization design (Roberts et al., 7 Aug 2025).

3. Perspectives, heuristics, and scoring logic

The core analytical apparatus of CDS consists of six perspectives and thirty heuristics, with five heuristics assigned to each perspective. The six perspectives are User, Environment, Interface, Components, Design, and Visual Marks (Roberts et al., 7 Aug 2025).

For the User perspective, the questions are: whether the design is suitable for the user and task; is understandable; does not require guesswork; is trustworthy; and would be useful. The associated anchors include “Unsuitable ↔ Suitable,” “Incomprehensible ↔ Understandable,” “Requires guesswork ↔ Clear assumptions,” “Distrustful ↔ Trustful,” and “Useless ↔ Useful” (Roberts et al., 7 Aug 2025).

For Environment, the questions ask whether the design fits with other technologies; uses suitable technology; has appropriate interaction; has correct sizing; and gives a positive ambience. The anchors include “Wrong setting ↔ Right setting,” “Unsuitable ↔ Right technology,” “Unsuitable ↔ Appropriate,” “Unsuitable ↔ Suitable size,” and “Poor ambience ↔ Positive” (Roberts et al., 7 Aug 2025).

For Interface, the heuristics address suitable user interface (GUI), ergonomic interface, suitably sized facets, suitable spacing, and the quantity of interface parts. The final item is anchored as “Too few/too many ↔ Suitable” (Roberts et al., 7 Aug 2025).

For Components, the heuristics assess whether the design has all necessary components, has all suitable view types, has clear relationships between parts, allows the task to be easily performed, and has good organisation of components. These are anchored by contrasts such as “Missing ↔ Complete,” “Unsuitable ↔ Suitable types,” “Unclear ↔ Clear relationships,” “Unfulfilled ↔ Easily performed,” and “Poor layout ↔ Good layout” (Roberts et al., 7 Aug 2025).

For Design, the questions examine whether the design is inspiring, aesthetic or visually attractive, has effective composition and data-space utilisation, provides suitable coverage of data or concepts, and contains clear labels or legends (Roberts et al., 7 Aug 2025).

For Visual Marks, the heuristics examine appropriate channel choice, correct data-to-mark mappings, suitable mark types, the right level of abstraction or zoom, and whether nothing is hidden incorrectly, with the final anchor stated as “Overplotting ↔ Clear” (Roberts et al., 7 Aug 2025).

This organisation makes CDS simultaneously broad and granular. It ranges from contextual fit and ergonomics to composition and mark-level representation. The paper characterises this breadth as covering “end-to-end design—from holistics to pixels” (Roberts et al., 7 Aug 2025). At the same time, the use of per-perspective scores means that local weaknesses can be surfaced even when the overall total is acceptable. The explicit warning that the average score may mask extreme weaknesses indicates that CDS is not meant to be interpreted as a single scalar verdict (Roberts et al., 7 Aug 2025).

4. First-impression vocabulary and Stage 1 elicitation

Stage 1 uses a controlled vocabulary of twenty first-impression adjectives. The set is: clear, confusing, sensible, indifferent, clever, reliable, pointless, indistinctive, complex, organised, moderate, spectacular, useless, average, bad, fulfilling, useful, fair, vague, beautiful (Roberts et al., 7 Aug 2025).

The selection procedure for these adjectives is specified. The vocabulary began with a card-sort of critical-thinking terms in a 2-day workshop in 2017. The terms were then grouped into positive, neutral, and negative using SentiWordNet scores; balanced to 7 positive, 6 neutral, and 7 negative; and refined by talk-aloud sessions to twenty terms, removing ambiguity (Roberts et al., 7 Aug 2025).

This part of CDS serves several functions. Factually, it supplies the $$S_1:\mathcal{A}\to P_1,\qquad S_2:\mathcal{A}\times P_1\to P_2,\qquad S_3:\mathcal{A}\times P_1\times P_2\to R$$8 from which the appraiser selects $$S_1:\mathcal{A}\to P_1,\qquad S_2:\mathcal{A}\times P_1\to P_2,\qquad S_3:\mathcal{A}\times P_1\times P_2\to R$$9. Contextually, it creates an explicit bridge between initial affective or rhetorical judgement and later heuristic analysis. A plausible implication is that the adjective selection externalises tacit judgement early, making later reflection less likely to obscure first impressions.

The emphasis on an indicative title and concise summary also matters methodologically. A 2–3 word title and 1–2 sentence summary constrain the appraiser to a compact articulation of what the artefact is trying to do before any detailed critique is performed (Roberts et al., 7 Aug 2025).

5. Development and refinement, 2017–2023

The paper describes a sustained process of refinement from 2017 to 2023, culminating in four main versions of CDS and continued classroom use (Roberts et al., 7 Aug 2025).

Version	Period	Reported refinements
1	2017–2019	Draft “critical-analysis sheet,” 20 adjectives, 6 perspectives, ~63 prompt words, 30 heuristics, star-plot visual summary
2	2019–2020	Simplified sheet layout, clarified wording, retained 20 words and 6×5 heuristics, star-plot removed
3	2020–2023	Semantic-differential word pairs, expanded support notes and lecture materials, rewritten web-app
4	2023–present	Final wording refinements, two sheet versions, integration in BSc/MSc modules since 2023

Version 1 emerged from a 2-day vocabulary workshop that produced a draft “critical-analysis sheet.” It included the card-sort leading to twenty adjectives, six perspectives, and approximately sixty-three prompt words. Talk-aloud sessions then distilled this material into thirty heuristics arranged as $$P_1=\{(Name, Summary, W_5)\mid Name\in Strings,\ Summary\in Strings,\ W_5\subseteq W,\ |W_5|=5\}$$0, and a star-plot visual summary was added (Roberts et al., 7 Aug 2025).

Version 2 simplified the sheet layout, removed colour, clarified wording, retained the twenty words, six-by-five heuristic structure, and 5-point Likert scale from $$P_1=\{(Name, Summary, W_5)\mid Name\in Strings,\ Summary\in Strings,\ W_5\subseteq W,\ |W_5|=5\}$$1 to $$P_1=\{(Name, Summary, W_5)\mid Name\in Strings,\ Summary\in Strings,\ W_5\subseteq W,\ |W_5|=5\}$$2, and removed the star-plot because it was too time-consuming manually (Roberts et al., 7 Aug 2025).

Version 3 introduced semantic-differential word pairs for each heuristic, expanded support notes and lecture materials, and completely rewrote the web-app for digital entry and archive (Roberts et al., 7 Aug 2025).

Version 4 brought final wording refinements of heuristics and instructions, produced two sheet versions with and without differential pairs, and integrated CDS in BSc/MSc modules since 2023 (Roberts et al., 7 Aug 2025).

The paper explicitly links these refinements to longstanding use in teaching each year. This sustained use is presented as enabling reflection on practical application and the provision of guidance for others adopting the method (Roberts et al., 7 Aug 2025). This suggests that CDS was not introduced as a static framework but developed through iterative pedagogical deployment.

6. Teaching use, adoption guidance, and limitations

Two teaching examples are provided to illustrate the method in practice. In an undergraduate eight-week poster project, a student titles the design “Population Flows,” summarises it as “A static poster showing migration over time between five countries,” and selects the adjectives $$P_1=\{(Name, Summary, W_5)\mid Name\in Strings,\ Summary\in Strings,\ W_5\subseteq W,\ |W_5|=5\}$$3. In Stage 2, the student scores all thirty questions; the User perspective is shown as $$P_1=\{(Name, Summary, W_5)\mid Name\in Strings,\ Summary\in Strings,\ W_5\subseteq W,\ |W_5|=5\}$$4, and Visual Marks include a $$P_1=\{(Name, Summary, W_5)\mid Name\in Strings,\ Summary\in Strings,\ W_5\subseteq W,\ |W_5|=5\}$$5 on question 26 because the colour scale is ambiguous. In Stage 3, the Total Score is reported as $$P_1=\{(Name, Summary, W_5)\mid Name\in Strings,\ Summary\in Strings,\ W_5\subseteq W,\ |W_5|=5\}$$6, interpreted as “OK but room to improve,” with Visual Marks and Environment as the weakest areas. The next steps are to refine the colour channel, add annotations, and ensure a clear legend (Roberts et al., 7 Aug 2025).

In a postgraduate interactive-tool example, the artefact is titled “Traffic VR Viz” and summarised as “A head-mounted VR tool showing live city-traffic patterns.” The selected adjectives are $$P_1=\{(Name, Summary, W_5)\mid Name\in Strings,\ Summary\in Strings,\ W_5\subseteq W,\ |W_5|=5\}$$7. The Environment score is shown as $$P_1=\{(Name, Summary, W_5)\mid Name\in Strings,\ Summary\in Strings,\ W_5\subseteq W,\ |W_5|=5\}$$8, with interaction marked awkward at question 8, and the Interface includes question 12 at $$P_1=\{(Name, Summary, W_5)\mid Name\in Strings,\ Summary\in Strings,\ W_5\subseteq W,\ |W_5|=5\}$$9 because it causes fatigue. The Total Score is $$P_2=\{Score(p,q)\in\{-2,-1,0,1,2\}\mid p\in P,\ q\in Q_p\}$$0, interpreted as indicating that the design needs redesign, with attention directed to ergonomics, interaction, and a simpler UI (Roberts et al., 7 Aug 2025).

The adoption guidance is explicit. Preparation and context require deep understanding of data, tasks, and users before Stage 1, together with a clear scenario framed by the “five W’s.” Process is emphasised over mere completion: CDS is described as being as much about structured thinking as about the final score, and users are encouraged to engage in slow, honest reflection rather than “just ticking boxes.” Repetition is recommended, with at least two CDS uses, such as a design plan and a final artefact, alongside lectures and in-class examples to build critical vocabulary (Roberts et al., 7 Aug 2025).

Tool support includes an online web-app hosted by the authors for digital scoring and history, printable PDF sheets with and without semantic differentials, and spreadsheet templates for automatic total and perspective sums (Roberts et al., 7 Aug 2025).

The stated benefits are that CDS scaffolds critical thinking for novices, covers end-to-end design from holistics to pixels, and generates actionable next steps. The stated limitations are that it is time-consuming on first use, approximately 10–15 minutes and then approximately 6 minutes; that the average score may mask extreme weaknesses; and that it can feel rigid if used too dogmatically, so it should always be interpreted alongside narrative. Extensions proposed in the paper include adapting heuristics for domain-specific needs such as BI dashboards and combining CDS with user testing or analytic metrics in a mixed-methods approach (Roberts et al., 7 Aug 2025).

Taken together, these details frame CDS as a heuristic evaluation method whose distinctive contribution lies in coupling first-impression elicitation, structured multi-perspective scoring, and explicit reflective synthesis. Its primary domain is visualisation design, but the paper’s guidance on adaptation and mixed-methods use indicates a broader methodological role as a repeatable critical-reflection scaffold within design education and practice (Roberts et al., 7 Aug 2025).