Optimal Mixing of Question Types in Visualization Assessments

Determine the optimal mixture of multiple-choice and alternative question types for assessing visualization knowledge and skills in visualization assessment and concept inventory research.

Background

The paper develops a 14-question multiple-choice concept inventory intended to assess the breadth of visualization design skills emphasized by instructors. The authors note that although multiple-choice formats are scalable and traditional for concept inventories, they are poorly suited to evaluating some abilities, particularly visualization construction. This leaves unresolved how different question formats should be combined to balance scalability with the assessment of practical and generative visualization skills.

References

Optimal mixing of question types is still an open discussion in both visualization assessment and concept inventory research, which could be a promising direction for future work.

— What Do Visualization Instructors Want Students to Learn? Introducing a Concept Inventory for Visualization Design  (2608.20090 - Lamkin et al., 20 Aug 2026) in Section 6, “Discussion and future work,” subsection “Limitations and Lessons Learned,” paragraph “Limitation: Multiple Choice Questions”