PatternSight: Chart Perceptual Grouping Tool

• PatternSight is a tool that assesses chart perceptual grouping based on Gestalt principles and interpretable visual-effect weights.
• It uses a 23-dimensional visual-effect vector and contrastive learning to predict which SVG elements form coherent groups based on position and appearance.
• The system provides an interactive interface for real-time design assessment, offering actionable recommendations to enhance chart clarity.

PatternSight is a chart-authoring support system for assessing whether a visualization is likely to make intended graphical patterns perceptually salient to viewers. It is centered on a perception simulation model that predicts which groups of SVG elements viewers are likely to notice as coherent graphical patterns, explains those predictions through interpretable visual-effect weights, and embeds the resulting analysis in an interactive interface for design assessment and revision. The system is motivated by the observation that chart effectiveness depends not only on encoding syntax or data structure, but on perceptual grouping phenomena such as proximity, similarity, connectedness, and common region; accordingly, PatternSight operationalizes perceptual grouping as a computational problem over chart elements and their visual attributes (Wang et al., 17 Jul 2025).

1. Problem setting and conceptual scope

PatternSight addresses a specific failure mode in contemporary chart authoring: a chart author may intend one data pattern to stand out, while viewers may instead perceive a different grouping, or may divide an intended group into multiple perceptual units. The system is designed for settings in which authors want to know not merely how data are encoded, but what graphical patterns viewers will likely notice first and why. This emphasis distinguishes perceptual effectiveness from purely syntactic correctness.

The underlying problem formulation is explicitly perceptual. A chart may support multiple possible groupings by color, proximity, shape, alignment, connectedness, or enclosed region. Existing authoring systems such as Voyager 2, Vega-Lite, Data Illustrator, Falx, Tableau, and PowerBI are described as strong at generating charts, applying templates, or recommending design choices, but not at dynamically assessing perceptual grouping effectiveness. PatternSight is proposed as a response to that gap, especially for authors without formal training in perceptual theory or for charts that are complex, dense, or multi-dimensional (Wang et al., 17 Jul 2025).

A central premise is that evaluating graphical patterns through human studies or interviews is too slow and expensive for routine authoring. PatternSight therefore substitutes an automated but interpretable simulation of visual grouping. This suggests a shift from rule-based visualization critique toward predictive perceptual assessment, where the object of analysis is the grouping behavior of viewers rather than the encoding specification alone.

2. Perception simulation model

At the core of PatternSight is a contrastive perception simulation model trained from human annotations of perceptual grouping. The model begins from perceptual grouping theory, especially Gestalt principles, and translates it into a structured feature representation of chart elements. The paper states that grouping is driven mainly by two kinds of visual effects: position and appearance. Position includes proximity, connectivity, and common region; appearance includes similarity in type, size, fill, stroke, and color (Wang et al., 17 Jul 2025).

To make these factors computable, each SVG element is encoded as a 23-dimensional visual-effect vector. The feature set includes shape or type, color in HSL space, size, stroke width, bounding-box coordinates, centroid position, and pairwise positional relationships. Because hue is cyclic, it is split into sine and cosine components. For relation-based features, the model computes pairwise relationship matrices and applies multidimensional scaling to derive descriptive dimensions, so that inter-element relations can be represented in a feature space comparable to other visual effects.

The training data were collected from 40 charts taken from D3.js and Highcharts examples. Thirty-five annotators labeled perceivable patterns in those charts, marking element groups and rating each pattern on intra-group consistency and boundary distinctness. The resulting corpus contains 1,273 pattern annotations. From these annotations, the model derives positive and negative pairwise relationships: positive pairs are element pairs inside a human-marked group with high consistency ratings, whereas negative pairs are sampled from outside the group, with sampling probability based on a Gaussian kernel over distance so that nearby competing elements are emphasized. The rationale given is that proximity can override other cues.

Training uses contrastive learning, motivated by the claim that human perception is fundamentally comparative. The model therefore learns an embedding in which perceptually grouped elements are near one another and non-grouped elements are separated. The output is not limited to binary grouping predictions. It also yields perceptual weights $\boldsymbol{w}$ over visual-effect dimensions, which make the prediction interpretable by indicating which dimensions matter for grouping (Wang et al., 17 Jul 2025).

3. Formalization of consistency, salience, and multiple outcomes

PatternSight formalizes grouping in terms of weighted feature similarity. Given an element $i$ with visual feature vector $v_i$, the model uses the weighted representation $|\boldsymbol{w}_i|\cdot v_i$. Element consistency is then defined as

$$C(i,j) = \cos\bigl(|\boldsymbol{w}_i|\cdot v_i,\; |\boldsymbol{w}_j|\cdot v_j\bigr).$$

For a candidate pattern $E$, salience is defined as the ratio of within-group consistency to between-group similarity:

$$S_E = \frac{\text{Avg}(C(i, j)\mid i\in E, j\in E, i\neq j)}{\text{Avg}(C(i, j)\mid i\in E, j\notin E)}.$$

Under this definition, a pattern is salient when its members are coherent with one another and distinct from non-members. The formulation explicitly combines internal homogeneity with external separability rather than optimizing either one in isolation (Wang et al., 17 Jul 2025).

The system is also designed to produce multiple perceptual outcomes. This follows from the observation that different viewers may perceive different plausible patterns in the same chart. PatternSight handles this by trimming the learned representation into distinct sub-representations, each corresponding to a different perceptual perspective. In practical terms, the system can surface several similar or overlapping patterns rather than a single dominant grouping.

Overlap analysis is built into the output stage. If two identified patterns share more than 80% of their elements, the system extracts the intersection as a core pattern and marks the pair as similar. This allows the interface to expose both stable common structure and near-duplicate alternative readings. A plausible implication is that PatternSight treats perceptual ambiguity as analyzable structure rather than as annotation noise.

4. Interface architecture and authoring workflow

PatternSight embeds the perception model in an interactive prototype oriented toward chart revision. The SVG Editor accepts an SVG file or source code from charting systems including ECharts, Vega-Lite, D3, and Highcharts, and provides basic editing operations such as reset, search, and replace. The use of SVG is central: the system operates directly on graphical elements rather than on abstract data semantics (Wang et al., 17 Jul 2025).

The visual elements view renders the chart and supports perceptual scoping. Authors can exclude irrelevant items such as legends, then select arbitrary groups by clicking, lassoing, or batch-selecting by element type. For a selected group, the system computes salience in real time. This turns the interface into an inspection instrument for hypothesized patterns as well as an exploration tool for model-discovered ones.

The patterns list presents all identified patterns as cards sorted by salience. Each card summarizes element types, contributing visual-effect dimensions, and the salience score. A compact overview uses color-coded squares, and related patterns are linked when they overlap substantially. This creates a navigation structure over alternative perceptual organizations of the same chart.

Two additional views make the system explanatory rather than merely predictive. The visual effect distribution view shows histograms for activated visual-effect dimensions and highlights the selected group’s proportion, allowing authors to inspect whether a group is internally consistent or visually diverse. The visual effect assessment view explains the current design and recommends specific improvements. It lists used and unused visual-effect dimensions, identifies conflicts among encodings, and proposes edits in three forms: adding a new dimension to strengthen grouping, modifying inconsistent visual effects within the selected group, or adding annotations such as enclosing regions or connecting lines. Recommendations are output in code-friendly form, including concrete SVG-level edits such as changing stroke color to rgb(234, 0, 0) (Wang et al., 17 Jul 2025).

This workflow positions PatternSight not simply as a ranking engine for chart designs, but as a diagnostic system that links perceptual outcomes to modifiable visual causes.

5. Empirical evaluation

The evaluation comprises both model validation and a user study of the interface. For model evaluation, the authors collected a separate validation set of 20 charts annotated by 20 participants, producing 812 annotations. Model output was compared against human annotations using three metrics: element group accuracy (EGA), pattern coverage rate (PCR), and association consistency (AC) (Wang et al., 17 Jul 2025).

The reported overall results are:

Metric	Reported value	Interpretation in the paper
EGA	0.860	Closeness of predicted groups to the most similar human pattern
PCR	0.843	Coverage of the range of human-perceived patterns
AC	0.771	Similarity of element co-membership structure

These results are interpreted as evidence that the model matches salient human groupings, covers diverse perceptual outcomes, and captures many of the associative rules used by viewers. The paper also reports failure cases, especially for charts with irregular shapes such as stacked area charts, where the boundary extraction mechanism based on bounding boxes is insufficient; in such cases PCR and AC drop substantially.

The user study involved 15 participants with experience using visualization tools but without formal perceptual-theory training. After a short tutorial, they completed a training task and a free exploration phase with think-aloud feedback. Participants reported that chart optimization before using PatternSight was largely trial-and-error because they lacked feedback on whether a design change improved grouping. The System Usability Scale score was 81.2, and participants stated that the click-to-apply suggestions and automatic code modification substantially reduced effort (Wang et al., 17 Jul 2025).

Two case studies illustrate operational use. In a bar-chart scenario, a user attempted to balance salience between two product categories. PatternSight showed that the red category was more salient because of fill hue and aligned bounding-box tops; the user then experimented with color changes and found a more balanced solution by reducing the salience of the stronger group rather than over-amplifying the weaker one. In an exoplanet discovery map, the intended green “habitable zone” pattern was diluted by a large enclosing background circle. PatternSight identified similar patterns and indicated that stroke-based changes were available and unused; by increasing stroke width and changing stroke color, the user raised salience from roughly 68 to about 80, making the target group more noticeable.

6. Limitations, implications, and relation to adjacent systems

The paper states several limitations explicitly. PatternSight currently relies on SVG-level visual information rather than underlying data or authoring code, which makes it weak at reasoning about encoding semantics such as exact scale mappings. It also struggles with irregular boundaries and complex shapes because of simplified boundary extraction. Future work is suggested in two directions: integrating computer vision models for better shape understanding and using authoring code to infer richer encoding logic (Wang et al., 17 Jul 2025).

These limitations clarify the scope of the system. PatternSight is a perceptual grouping assessor, not a general semantic verifier of visualization correctness. Its strengths lie in theory-grounded feature construction, limited-data learning, interpretability, and support for iterative authoring. This suggests that its most appropriate use is as a perceptual design aid rather than as a complete chart recommendation engine.

Within the broader landscape of pattern-oriented visualization systems, PatternSight occupies a distinct position. ShapeSearch, for example, addresses shape-based exploration of trendlines through sketch, natural-language, and visual regular-expression queries, supported by a shape querying algebra and perceptually aware scoring (Siddiqui et al., 2018). PatternSight does not perform trendline retrieval; instead, it predicts perceptual grouping outcomes in charts already under design. The difference is methodological as well as functional: ShapeSearch supports explicit query specification over line shapes, whereas PatternSight models likely viewer perception over arbitrary chart elements.

A broader comparison can also be drawn to systems that search graphical patterns in other domains. Online search over electronic band structures in the Organic Materials Database uses sliding windows, cosine distance, and approximate nearest-neighbor indexing to retrieve user-specified graphical motifs in band plots (Borysov et al., 2017), while convolutional pattern spotting in historical documents uses RetinaNet feature-pyramid embeddings and query-by-example retrieval to localize repeated graphical objects in manuscript pages (Úbeda et al., 2019). PatternSight differs from both in that its target is not general graphical retrieval but perceptual grouping effectiveness in authored charts.

Taken together, these comparisons indicate that PatternSight belongs to a wider research movement that treats patterns as first-class computational objects, but its distinctive contribution is to make perceptual grouping itself the object of prediction, explanation, and interactive design intervention.