MathCanvas-Instruct: Unified Math Reasoning

• MathCanvas-Instruct is a large-scale, richly annotated dataset that interleaves textual reasoning with strategic visual actions to solve mathematical problems.
• It employs a rigorous four-stage construction pipeline with quality controls like deduplication, image standardization, and domain categorization to ensure high-quality, interleaved reasoning.
• The framework significantly boosts multimodal model performance by embedding an explicit visual chain-of-thought, leading to substantial gains in complex math problem solving.

MathCanvas-Instruct is a large-scale, richly annotated dataset and instructional framework designed to train and evaluate unified Large Multimodal Models (LMMs) in mathematical domains that require intrinsic, interleaved visual and textual reasoning. It is the central training resource for Phase 2 (“Strategic Visual-Aided Reasoning”) of the MathCanvas system, teaching models not only to generate mathematical diagrams but to strategically decide when and how to invoke visual aids within stepwise chains of deduction. MathCanvas-Instruct comprises 219,000 expert-curated examples in which each textual reasoning step is interleaved with precisely-timed visual actions, forming a comprehensive blueprint for grounded multimodal mathematical problem solving (Shi et al., 16 Oct 2025).

1. Conceptual Foundation and Purpose

MathCanvas-Instruct addresses a core bottleneck in multimodal mathematical reasoning: previous LMMs and chain-of-thought protocols operated primarily on linear text, with visual content appended as static, extrinsic snapshots that failed to capture the dynamism and strategic use of diagrams in authentic mathematical practice. MathCanvas-Instruct advances beyond this by endowing models with an explicit, stepwise “Visual Chain-of-Thought” (VCoT), teaching them the high-level decision process (“Should I draw now?”) and the low-level execution (“What exactly should be drawn to make further progress?”). Unlike prior datasets focused on isolated diagram generation or text-only deduction, MathCanvas-Instruct pairs every meaningful inference step with either free-form LaTeX-rich textual explanation or a carefully delimited visual operation (Shi et al., 16 Oct 2025).

This design enables LMMs to emulate the human process of visual-aided mathematical insight, particularly for problems in geometry, calculus, statistics, and related fields that intrinsically require diagrammatic reasoning. The dataset explicitly encodes both the necessity and strategic timing of each diagram, ensuring that multimodal reasoning is not merely decorative but essential for task resolution.

2. Dataset Construction and Quality Control

The construction pipeline for MathCanvas-Instruct consists of four rigorous, computationally intensive stages operating on an initial set of 632,000 multimodal mathematics problems sourced from textbooks, examinations, and educational websites:

1. Visual and Logical Filtering: GPT-5 removed any example where the accompanying image was irrelevant to the deductive process, standardizing all inline formulas to LaTeX, reducing the set to 367,000.
2. Answer Verification and Diagram Quality: A subsequent GPT-5 pass eliminated entries with missing or incorrect answers, ambiguous or low-resolution diagrams, and non-reasoning “draw the figure” tasks, yielding 303,000 high-quality pairs.
3. Deduplication and Image Standardization: Problems were deduplicated by 5-gram overlaps for text and perceptual image hashing, yielding 222,000 unique pairs. Images were standardized to 512×512 resolution using SwinIR-based super-resolution.
4. Hierarchical Categorization and Benchmark Split: GPT-4.1 assigned all samples to eight mathematical domains (Algebra, Analytic Geometry, Calculus & Vectors, Plane Geometry, Solid Geometry, Statistics, Transformational Geometry, Trigonometry). 3,000 problems formed the MathCanvas-Bench benchmark; the remaining 219,000 constitute the final MathCanvas-Instruct corpus.

Quality assurance was maintained by extensive automated filtering, deduplication, and human spot checks (covering at least 5% of the data), ensuring logical coherence and genuine necessity for each visual step. Explicit answer tagging, LaTeX embedding, and diagram-format standardization guarantee unambiguous parsing and evaluation (Shi et al., 16 Oct 2025).

3. Structure and Composition of Examples

Each MathCanvas-Instruct example is composed of:

• Input Block: A problem statement in text (including embedded LaTeX) and, where present, one or more input diagrams.
• Interleaved Reasoning Sequence: An ordered chain combining natural language explanation segments and discrete visual steps. Visual operations are encoded as special tokens <im_start> ... <im_end> containing precise drawing or editing instructions. All visual steps correspond to diagrams captured at 512×512 resolution. Every chain is capped at five visual steps.
• Explicit Answer Markup: Final answers and sub-answers are tagged via explicit markers (e.g., \<1>...</1>) for consistent automated extraction.

The following table summarizes dataset composition characteristics:

Attribute	Value / Distribution
Total examples	219,000
Multimodal (diagram-necessary)	65%
Text-only problems	35%
Grade coverage	63% middle (7–9), 37% high (10–12) school
Avg. question length	108 tokens (max 466)
Avg. solution length	540 tokens (max 2001)
Avg. input images	1.03 per multimodal problem
Avg. generated images in solution	1.18 (up to 5)
Number of domains	8 (see above)

Problems are predominantly multi-step: while 68% require a single final answer, 18% involve two sub-questions, 12% three, and 2% four or more (Shi et al., 16 Oct 2025).

4. Reasoning Protocol and Annotation Guidelines

All MathCanvas-Instruct solutions were authored under a strict style guide:

• Each visual action is included only if it unlocks a logical progression; superfluous or decorative diagrams are forbidden.
• No more than five incremental diagram edits per problem; each is tightly coupled to an explanatory step.
• Algebraic, trigonometric, and symbolic expressions are fully embedded in LaTeX to enable mathematical precision and standardization.
• Final answers are enclosed in explicit tags for deterministic evaluation.
• Visual aesthetics (line, font, color) are controlled to assure clarity and consistency.

This annotation protocol ensures that the dataset does not conflate low-level drawing skills with strategic, content-driven visual reasoning. Human spot checks confirm the logical co-dependence between each diagram and its adjacent textual inference (Shi et al., 16 Oct 2025).

5. Evaluation, Ablations, and Effectiveness

Although MathCanvas-Instruct is designed as a training corpus rather than a standalone benchmark, its impact is measured by downstream improvements on MathCanvas-Bench—a challenging set of 3,000 interleaved visual-textual problems. Key evaluation metrics and findings include:

• Models trained without MathCanvas-Instruct achieve a weighted MathCanvas-Bench score of 18.7%.
• Fine-tuning on MathCanvas-Instruct increases performance to 34.4%, yielding an absolute gain of 15.7 points (an 84% relative improvement over the text-only baseline).
• Ablation studies show that removing interleaved visual steps reduces performance by 3.5 weighted points, demonstrating the critical value of precise, well-timed diagram integration (Shi et al., 16 Oct 2025).

This training-to-evaluation pipeline establishes that stepwise diagrammatic deduction, as encoded in MathCanvas-Instruct, is essential for human-level performance in visually grounded mathematical reasoning.

6. Connections to Canvas-Based and Interactive Reasoning Systems

MathCanvas-Instruct shares conceptual lineage with several complementary multimodal reasoning architectures:

• Canvas-of-Thought (Canvas-CoT) introduces a persistent HTML5 canvas on which the LLM performs atomic CRUD (Create, Replace, Update, Delete) operations on geometric primitives expressed as DOM nodes. This externalized, addressable state enables efficient, in-place corrections of diagrams, reduces context overhead, and supports a rendering-based critique loop for visual constraint enforcement. MathCanvas-Instruct can leverage this paradigm by mapping its <im_start>...</im_end> commands to CRUD actions and DOM-based state updates, ensuring that each visual operation is incremental and revisable (Sun et al., 11 Feb 2026).
• Wisdom Computing’s step-by-step animation system employs AI-driven handwriting recognition (YMCANet = YOLOv11n + Mamba + CoordAttention), followed by DBSCAN-based matrix grid mapping and Manim-powered instructional animation. For MathCanvas-Instruct, this pipeline enables live user input (via HTML5 Canvas), real-time parsing and LaTeX rendering, dynamic instructional animations, plugin-based extensibility for new math operations, and user-feedback correction loops, all underpinned by high-accuracy detection and modular scene management (Yu, 2 May 2025).
• Interactive Sketchpad implements a bidirectional tutoring loop pairing LMM-based text and code-generated visuals with an interactive whiteboard interface. This supports dialogic reasoning, freeform sketch-based interaction, feedback on both AI- and human-generated diagrams, and a “hint-first” instructional approach. For MathCanvas-Instruct, these patterns suggest modular separation of analysis, code-driven diagram synthesis, sketch interpretation, and transparent feedback mechanisms—all supported by code execution sandboxes and visual reasoning triggers (Chen et al., 12 Feb 2025).

This suggests that MathCanvas-Instruct’s representational and operational format is compatible with both persistent canvas-based CRUD frameworks and dynamic, animation-driven pedagogical systems, providing a maximal substrate for both model inference and interactive, real-time human-AI collaboration.

7. Implications and Future Directions

MathCanvas-Instruct establishes a new paradigm for training LMMs in mathematically grounded, visual-textual reasoning. Its impact is underscored by dramatic gains in MathCanvas-Bench performance and robust ablation evidence for the necessity of strategic diagram interleaving. Practical implications include:

• Blueprint for multimodal curriculum construction: MathCanvas-Instruct’s protocol can inform the development of future datasets targeting other STEM domains where dynamic, interleaved visual reasoning is essential.
• Compatibility with differentiable and stateful substrates: Its design aligns naturally with CRUD-enabled canvas protocols and incremental animation frameworks, facilitating efficient feedback loops, error correction, and live instruction.
• Evaluation and generalization benchmarks: MathCanvas-Bench serves as a rigorous, held-out standard for quantifying model advances and diagnosing specific failures in multimodal, stepwise mathematical problem solving.

A plausible implication is that as LMMs adopt increasingly fine-grained, interactive visual reasoning architectures, the interleaved, high-fidelity, and need-driven design of resources like MathCanvas-Instruct will be central to bridging the gap between human intuition and machine deduction, especially in fields where visual context is indispensable for inference (Shi et al., 16 Oct 2025, Sun et al., 11 Feb 2026, Yu, 2 May 2025, Chen et al., 12 Feb 2025).