Energy and Momentum Conceptual Survey

• EMCS is a rigorously developed 25-item assessment tool that probes students’ conceptual grasp of energy and momentum principles in mechanics.
• It employs detailed distractor analysis and statistical validation to evaluate understanding of work–energy and impulse–momentum theorems.
• The survey provides actionable diagnostic feedback, supporting targeted instructional interventions and advancing physics education research.

The Energy and Momentum Conceptual Survey (EMCS) is a rigorously constructed, expert-validated 25-item multiple-choice instrument designed to probe the qualitative understanding of introductory physics students with respect to core energy and momentum concepts. The instrument is extensively used in physics education research to diagnose student difficulties, compare instructional modalities, and trace conceptual progression from introductory to upper-division courses (Singh et al., 2016, Singh et al., 2016, Brundage et al., 2023, Savage et al., 20 Aug 2025). The structure, development methodology, statistical properties, and instructional implications of the EMCS are detailed below.

1. Test Structure and Conceptual Domains

The EMCS is structured to comprehensively assess conceptual knowledge in mechanics by focusing on four foundational principles: the work–energy theorem, conservation of mechanical energy, the impulse–momentum theorem, and conservation of linear momentum. These are distributed across the item pool as follows (Singh et al., 2016, Brundage et al., 2023):

Topic	Number of Items	Example Concepts
Energy	14	Work (by conservative/non-conservative forces), mechanical energy conservation, work–energy theorem ($W = \Delta K$)
Momentum	10	Impulse–momentum theorem ($\vec{p}_f - \vec{p}_i = \int \vec{F}\,dt$), momentum conservation (elastic/inelastic collisions)
Integrative	1	Contexts requiring both energy and momentum principles

All items target the "interpretation" and "inference" levels of Bloom’s taxonomy, minimizing rote recall and emphasizing deeper conceptual reasoning. Representative items include probes of gravitational work path independence, correct identification of the system for conservation principles, and nuanced collision scenarios requiring the application of both momentum and energy considerations.

2. Survey Development and Validation Process

The EMCS was constructed via a multi-phase process:

1. Blueprinting and Free-Response Baseline: Content scope and cognitive complexity were determined by faculty through a content matrix and performance rubric. Fifty free-response questions were administered to hundreds of students, and ten individual think-aloud interviews were conducted using established protocols to surface common reasoning patterns (Singh et al., 2016).
2. Item Writing and Expert Review: Distractors were systematically drawn from robust misconceptions observed in the qualitative baseline. Ten physics faculty and postdoctoral fellows reviewed item drafts for clarity, appropriateness, and difficulty, informing iterative revisions.
3. Pilot Testing, Classical Item Analysis, and Revision: A 28-item pilot was administered to ~500 students. Metrics calculated included item difficulty (fraction correct, $p$), point-biserial discrimination indices (range: 0.21–0.48), and distractor analysis by quartile. The test was then refined to a 25-item format that demonstrated statistically acceptable psychometric properties (Singh et al., 2016).
4. Ongoing Validation: Multiple semesters of administration (N > 3000, across ~30 institutions) enabled further analysis and minor editorial revision after each cycle. Internal consistency reached $\alpha \approx 0.75$ for calculus-based, $\alpha \approx 0.68$ for algebra-based, and $\alpha > 0.80$ for graduate students—supporting reliability. Content and criterion-related validity were established via expert reviews and moderate correlation with final exam performance ($r\approx 0.6$) (Singh et al., 2016).

3. Statistical Properties and Scoring

The EMCS’s psychometric indices satisfy accepted measurement standards (Singh et al., 2016):

• Item Difficulty: $p$ ranged from 0.20 to 0.80; mean overall score for calculus-based students was ≈49.2% (SD = 18%), with energy-only ≈45.7% (SD = 20%), and momentum-only ≈55% (SD = 22%).
• Discrimination: Point-biserial discrimination typically >0.2 for all items; lowest observed PBD was 0.21, indicating good student separation.
• Reliability: Cronbach’s $\alpha$ exceeded 0.70 for all large cohorts; graduate students achieved $\alpha > 0.80$.
• Validity: Repeated expert review and matched content with standard texts (Halliday-Resnick-Walker, Taylor) and research-based tutorials provided content validity. Concurrent validity was demonstrated, as graduate students scored >80% while introductory students averaged <50% (Brundage et al., 2023).

The scoring rubric allots 1 point per correct response (no partial credit); energy and momentum subscores may be separately reported, with the integrative item apportioned as needed.

4. Probed Concepts, Persistent Difficulties, and Diagnostic Use

The EMCS uncovers widespread, persistent conceptual difficulties that remain largely robust even after traditional instruction:

• Energy Cluster: Students frequently treat work and energy as vectors, believe that gravitational work is path-dependent, and confuse the role of mass in gravitational acceleration and mechanical energy transformations (e.g., only 24% recognized that equal-height slides yield equal speeds) (Brundage et al., 2023, Singh et al., 2016).
• Momentum Cluster: Conceptual errors include failing to identify the correct closed system, over-attributing conservation to subsystems, conflating impulse and force, and misunderstanding elastic vs. inelastic collision outcomes. Approximately 40% believed momentum is conserved for each object separately in a collision (Singh et al., 2016, Singh et al., 2016).
• Context Sensitivity and Sequential Reasoning: Students often struggle with problems needing the sequential application of multiple principles (e.g., ballistic pendulum) and with transferring formal knowledge to novel contexts.
• Upper-Level Persistence: For every EMCS item where fewer than 50% of introductory students were correct post-instruction, fewer than two-thirds of upper-level students succeeded; 13 of 25 items met this "struggle" criterion even for physics majors after junior-level mechanics (e.g., only 62% correctness on inelastic collision + energy item at the upper level; 29% on projectile speed vs. launch angle) (Brundage et al., 2023).

The EMCS’s distractor analysis and subscore reporting enable diagnostic identification of such misconceptions, informing targeted intervention.

5. Innovations in Assessment: Open-Ended Responses and LLM Analysis

Recent research has leveraged LLMs for automated, scalable analysis of open-response EMCS items, supplying deeper resolution of student reasoning beyond fixed-choice distractors (Savage et al., 20 Aug 2025). LLMs, validated against human raters (Cohen’s $\vec{p}_f - \vec{p}_i = \int \vec{F}\,dt$0 = 0.65–0.89; 0–3% discrepancy), reliably categorized written explanations:

Question (Sample)	LLM-Emergent Error Categories (Incorrect Explanations)
Bullet–block collision (Q5)	Misapplying collision type, misapplying momentum, incorrect use of energy/work, Newton’s law confusion (67% incorrect overall)
Inelastic collision & ascent (Q16)	Omitting energy or momentum conservation steps, misapplying principles, impulse confusion (73% incorrect)
Impact force comparison (Q23)	Misapplication of momentum, force, or impulse concepts, collision mischaracterization (55% incorrect)

LLM-based classification revealed errors not directly paralleled in MC distractors (e.g., misapplying versus misidentifying collision type), thus exposing subtler reasoning failures. This suggests a new direction for assessment design and feedback: using emergent error categories to drive instructional redesign and to more effectively scaffold student understanding (Savage et al., 20 Aug 2025).

6. Instructional Impact and Recommendations

Empirical findings from EMCS deployments across multiple institutions demonstrate that:

• Traditional, quantitatively focused instruction in both introductory and upper-level settings fails to remediate many deep-seated conceptual misunderstandings. Normalized gains on the EMCS ($\vec{p}_f - \vec{p}_i = \int \vec{F}\,dt$1 for introductory, $\vec{p}_f - \vec{p}_i = \int \vec{F}\,dt$2 for upper-level) are low; typical post-test averages are 40–60% (Brundage et al., 2023, Singh et al., 2016).
• Active engagement and honors sections outperform standard lecture sections, with gains $\vec{p}_f - \vec{p}_i = \int \vec{F}\,dt$3 and higher overall/post-test means (Singh et al., 2016).
• The EMCS’s format provides actionable feedback at the item level, facilitating targeted remediation such as focused tutorials on conservation principles, system identification, or collision analysis.

Instruction is most effective when conceptual diagnostics are embedded alongside quantitative problem-solving, making explicit reference to the underlying system definitions and chain of physical principles required for correct inference. Recommendations emphasize deploying the EMCS as both pre- and post-test, analyzing subscore patterns, and integrating research-based curricular interventions to develop expert-like energy and momentum reasoning (Brundage et al., 2023, Singh et al., 2016).

7. Role in Physics Education Research

The EMCS is established as a statistically robust, content-validated diagnostic instrument extensively cited in the physics education research literature (Singh et al., 2016, Brundage et al., 2023, Singh et al., 2016, Savage et al., 20 Aug 2025). Its modular structure, nuanced item and distractor design, and demonstrated sensitivity to both instruction and persistent conceptual barriers make it central to studies of student learning progression and the effectiveness of curricular innovations.

A plausible implication is that conceptual mastery of energy and momentum requires systematic, explicit integration of the associated reasoning patterns across the full physics curriculum, not solely in introductory sequences. The EMCS's dual use as an assessment and research tool continues to shape both pedagogical practice and the empirical understanding of disciplinary learning trajectories.