Design Fixation Score (DFS) Metric

• Design Fixation Score (DFS) is a quantitative metric that measures the proportion of pre-defined design features reproduced in sketches, defining design fixation.
• It employs systematic coding with dual rater evaluations and Bayesian multilevel regression to robustly analyze feature reuse across ideation tasks.
• Higher DFS values, especially in AI-assisted conditions, correlate with reduced divergent thinking and creativity, underscoring practical constraints in design exploration.

Design Fixation Score (DFS) is a quantitative metric used to operationalize and measure design fixation in visual ideation tasks. Design fixation, in this context, is defined as the unintentional conformity towards existing ideas or concepts that constrains exploration of the design space. DFS is calculated as the proportion of an exemplar’s pre-specified salient features that re-appear in a participant’s generated sketch. This approach provides a structured means for evaluating the extent to which exposure to examples—such as those generated by generative AI or retrieved via image search—constrains creative output, both in terms of feature reuse and divergence from initial prompts (Wadinambiarachchi et al., 2024).

1. Conceptual Basis and Definition

Design fixation is characterized by reduced exploration and creativity, manifesting as the reproduction of features from known examples. The DFS operationalizes this phenomenon by quantifying how many “fixating features” from a provided example are present in a participant’s response. In applied settings, these features are denotative and discrete (e.g., limbs, antennae), and are pre-identified by researchers. The DFS reflects the fraction of example features reused and hence serves as a canonical metric for measuring fixation effects in design ideation studies.

2. Mathematical Formulation

The DFS is formalized as follows. Let $K$ denote the total number of pre-defined, salient (“fixating”) features in the example (e.g., $K = 14$ for a chatbot avatar with features such as arms, antennae, and large round eyes), and let $k_i$ represent the number of these features reproduced in sketch $i$. The Design Fixation Score for a given sketch is:

$\mathrm{DFS}_i = \frac{k_i}{K}$

where $\mathrm{DFS}_i$ ranges from 0 (no features reused) to 1 (all features reused). In empirical studies, individual DFS values may also be averaged across all sketches produced by a participant to yield a participant-level DFS (Wadinambiarachchi et al., 2024).

3. Coding Procedure and Reliability

DFS measurement requires systematic coding of the presence or absence of each feature in each participant’s sketch. Two trained, condition-blind raters independently evaluate scanned sketches in randomized order. For each sketch, both raters check which of the $K$ features are present. DFS values are then averaged across raters:

$\mathrm{DFS}_i = \frac{k_{1i} + k_{2i}}{2K}$

where $k_{1i}$ and $k_{2i}$ are the counts of features checked by each rater for sketch $K = 14$0. Inter-rater reliability is validated by comparing raw feature counts, with agreement statistics reported as $K = 14$1 across all sketches, indicating excellent coding reliability under standard psychometric thresholds. No formal Cohen’s $K = 14$2 or Krippendorff’s $K = 14$3 was reported (Wadinambiarachchi et al., 2024).

4. Experimental Workflow and Data Collection

The experimental workflow for DFS measurement involves several phases:

1. Viewing the Example: Each participant views an example (e.g., a chatbot avatar) with its 14 salient features visually highlighted.
2. Ideation Task: Participants generate as many divergent sketches as possible in a time-constrained session (e.g., 20 minutes).
3. Condition Variants:
   • Baseline: Sketches are generated from memory of the example.
   • Image Search: Participants can pause sketching to seek inspiration via Google Image Search.
   • GenAI: Participants can pause to write prompts and view Midjourney AI-generated images.
4. Sketch Scanning: All participant sketches are scanned and assigned unique IDs.
5. Coding: Two independent coders assess the presence/absence of each feature in every sketch.
6. DFS Calculation: DFS is computed per sketch and participant-level DFS is obtained by averaging across an individual’s submissions (Wadinambiarachchi et al., 2024).

5. Statistical Analysis and Model Interpretation

DFS distributions are modeled using Bayesian multilevel binomial regression with a probit link, accounting for hierarchical structure via random effects for participant and sketchID. The primary model specification is:

• $K = 14$4
• $K = 14$5

Posterior mean estimates and 89% credible intervals are provided for all parameters. Substantively, predicted mean DFS values for each condition are: Baseline ≈ 0.26, Image Search ≈ 0.33, and GenAI ≈ 0.36. Both image search and GenAI conditions yield reliably higher fixation than the baseline (Bayes Factors: Image Search ≈ 42.5, GenAI ≈ 124), with GenAI producing the largest increase (Wadinambiarachchi et al., 2024).

Condition	Model-predicted DFS	Bayes Factor (vs. Baseline)
Baseline	0.26	–
Image Search	0.33	≈ 42.5
GenAI	0.36	≈ 124

This analytic approach demonstrates heightened fixation in assisted ideation settings and quantifies the magnitude of these effects.

6. Relationship to Divergent Thinking and Creativity Outcomes

Elevated DFS is consistently associated with diminished novelty metrics. Experimental results indicate that higher DFS (in GenAI and Image Search conditions) correspond with lower fluency, variety, and originality in output. Although no direct correlations were formally computed between DFS and creativity indices across all sketches, the pattern of results evidences co-occurrence of high design fixation and reduced divergent-thinking performance. Within the GenAI subgroup, a Spearman rank correlation ($K = 14$6) confirmed that high fixation on AI-generated images predicted higher fixation in the subsequent user-generated sketch, corroborating a direct propagation of feature repetition from AI outputs to human ideation (Wadinambiarachchi et al., 2024).

7. Limitations and Methodological Considerations

The DFS metric, as implemented, presents several constraints:

• DFS captures only denotative, surface-level features, omitting connotative aspects such as style, emotional expression, or color palette, thus potentially underestimating fixation in these dimensions.
• Standard DFS does not measure “fixation displacement,” in which focus transfers from the example to new AI-generated imagery; qualitative analysis revealed that participants sometimes became fixated on the AI-generated examples themselves, escaping the influence of the original exemplar.
• While inter-rater reliability for feature coding was high, no formal Cohen’s $K = 14$7 or Krippendorff’s $K = 14$8 was reported; such statistics are recommended for greater transparency.
• The sample consisted of novice or moderate-level sketchers; generalizability to expert designers may be limited.
• The restricted 20-minute session likely induced trade-offs between active sketching and inspiration-seeking, possibly inflating fixation in tool-supported conditions. The dynamics of fixation may differ in real-world, longer-term ideation workflows involving generative AI (Wadinambiarachchi et al., 2024).

8. Prospects for Extension

Future research avenues include augmenting the DFS approach to accommodate richer, multidimensional indicators of fixation, encompassing both denotative and connotative elements. Explicit modeling of fixation displacement—quantifying the degree to which participants become anchored not only to provided exemplars but also to newly generated AI outputs—presents a prospective extension of the metric’s scope. Enhanced coding reliability procedures and application to expert samples are additional priorities for developing the metric’s generalizability and methodological rigor.

(Wadinambiarachchi et al., 2024)