MathCanvas: Visual Math Reasoning

• MathCanvas is a framework that integrates intrinsic Visual Chain-of-Thought by interleaving textual reasoning with high-fidelity diagrams for complex mathematical problem-solving.
• It uses a two-stage training paradigm, including diagram-centric pre-training and strategic fine-tuning on visual-textual datasets, significantly improving performance on benchmarks.
• The architecture features dual-expert Transformers and ViT/VAE pipelines that enable dynamic diagram generation and editing, aligning model reasoning with human expert workflows.

MathCanvas is a framework and system architecture developed to endow Large Multimodal Models (LMMs) with intrinsic Visual Chain-of-Thought (VCoT) reasoning for mathematics and related high-dimensional domains, notably geometry, trigonometry, and calculus. It structures both the model’s inputs and outputs as interleaved sequences of textual reasoning steps and high-fidelity, manipulable diagrams, enabling the model to natively determine when, what, and how to visualize as part of its deductive process. The MathCanvas paradigm departs from prior approaches that rely on rigid external toolchains or purely linear text-based chains, introducing a two-stage pre-training and fine-tuning paradigm, curated large-scale datasets, and a benchmark (MathCanvas-Bench) expressly targeting complex visual-textual integrated mathematical reasoning (Shi et al., 16 Oct 2025). Related developments include interactive, code-driven diagram platforms such as Interactive Sketchpad (Chen et al., 12 Feb 2025), which have converged in terminology with MathCanvas but are architecturally distinct.

1. Definition and Core Principles

MathCanvas refers to a two-phase training and deployment methodology for LMMs that enables “intrinsic VCoT”: the model’s native, autonomous capacity to decide when to initiate drawing actions, what diagrammatic elements advance the reasoning state, and how to refine or edit visual aids as it progresses through multi-step mathematical or geometric problem-solving (Shi et al., 16 Oct 2025). A distinguishing feature is the integration of image generation/editing and language modeling within a single LMM backbone, without requiring the mediation of external code interpreters or procedural toolchains that disrupt the model’s state. This results in seamless interleaving of text and diagrams, mirroring human expert workflows.

Intrinsic VCoT is operationalized through the alternation of two modalities:

• Text segments: Natural-language deduction, explanation, or symbolic manipulation.
• Diagram actions: Diagram build/edit operations, represented as images decoded from latent tokens or as explicit graphical modifications (e.g., draw, annotate, add auxiliary features).

2. Two-Stage Training Paradigm

Stage I: Visual Manipulation Pre-training

MathCanvas first pre-trains an LMM (BAGEL-Canvas architecture) on a large-scale, curated dataset focused exclusively on diagram generation and editing. The MathCanvas-Edit corpus (5.2M step-by-step editing trajectories) is sourced from expert-annotated geometry problems and synthetic sampling of geometric primitives and relations, ensuring both realistic and diverse coverage. Complementary to this, MathCanvas-Imagen comprises 10M caption-to-diagram pairs, spanning code-derived, public, and editorially curated sources.

The Generation Expert of the model is solely trained during this stage, using a rectified-flow objective on latent image representations:

$$L_\mathrm{flow} = \mathbb{E}_{t,\,\mathbf{x}_0,\,\boldsymbol{\epsilon}}\left[\left\| \hat{\mathbf{v}}_\theta(\mathbf{x}_t, t) - (\boldsymbol{\epsilon} - t\nabla_{\mathbf{x}_t}\log p(\mathbf{x}_t)) \right\|_2^2\right]$$

No text cross-entropy loss is applied at this stage (Shi et al., 16 Oct 2025).

Stage II: Strategic Visual-Aided Reasoning Fine-Tuning

Fine-tuning is conducted on the MathCanvas-Instruct dataset (219K examples), which contains carefully curated interleaved sequences of text and diagram edits. The token-level predictor is trained to determine, at each step, whether to proceed with text or initiate a drawing/editing action ($<$|vision_start|$>$ vs $<$|endoftext|$>$). Loss is a weighted sum of cross-entropy and rectified-flow terms:

$$L = \lambda_{\mathrm{CE}}L_\mathrm{CE} + \lambda_\mathrm{flow} L_\mathrm{flow}, \quad \lambda_{\mathrm{CE}}=0.25, \;\lambda_\mathrm{flow}=1.0$$

All model parameters are unfrozen. Contextual information includes input problem images encoded with a ViT, and solution images are noised during decoding to encourage robust visual reasoning. The architecture integrates a dual-expert Transformer (Understanding and Generation), classifier-free guidance, and explicit ViT/VAE pipelines.

3. Benchmarking: MathCanvas-Bench

MathCanvas-Bench is a specialized evaluation set of 3,000 generative, open-ended mathematics problems, systematically sampled to span eight core categories (Algebra, Analytic Geometry, Plane Geometry, Solid Geometry, Trigonometry, Statistics, Calculus & Vectors, Transformation Geometry). Each problem demands an interleaved visual-textual solution path—not mere answer boxes or multiple choice.

Evaluation metrics are:

• Complete Accuracy: Full credit only if all subparts are correct.
• Weighted Scoring: Each sub-question $i$ in $N$-part problems receives a weight

$w_i = \frac{1.3^{i-1}}{\sum_{j=1}^N 1.3^{j-1}}$

emphasizing correct completion of later, typically harder, substeps.

Automated assessment leverages a GPT-4.1-based answer extractor to ensure consistent scoring (Shi et al., 16 Oct 2025).

4. System Architecture and Internal Representations

MathCanvas is instantiated in the BAGEL-7B-MoT architecture, which incorporates:

• Bimodal Transformer Experts: Decoupled for understanding (comprehension, question parsing) and generation (diagram synthesis/editing).
• ViT encoder and VAE decoder: For mapping input/output images to and from latent representations.
• Classifier-free guidance: For stable multimodal generation.
• Token-based drawing control: Enabling atomic, low-level diagram edits aligned and temporally interleaved with text segments.

Internally, mathematical diagrams are structured as compositional sets of primitives (e.g., points, line segments, arcs, annotated angles), with editing trajectories reflecting the procedural semantics of mathematical problem-solving. Editing actions are referenced in pseudo-code and natural mathematical notation (e.g., “Add perpendicular from $B$ to line $<$0”, or “Draw $<$1 as the circle’s center, then add chord $<$2”).

5. Applications: Visual-Textual Mathematical Reasoning

MathCanvas’s core application domain is visual mathematical problem-solving, where diagrammatic manipulation is integral to precise and effective reasoning. Sample solution paths illustrate native VCoT capabilities:

• Foot of Perpendicular in Triangle Geometry: Sequentially constructs triangle $<$3, computes and draws the foot $<$4 of the perpendicular from $<$5 to $<$6, deduces $<$7 using the Law of Cosines, and visually labels the result.
• Auxiliary Circle in Angle/Arc Derivation: Constructs a circle, adds chord $<$8, derives and annotates $<$9, then computes and visualizes $>$0 for arc length.

This interleaving enables the model to select and construct visual aids when human experts would do so, rather than at arbitrary or post-hoc stages (Shi et al., 16 Oct 2025).

Related systems such as Interactive Sketchpad operationalize similar principles via Python code execution to generate dynamic diagrams, embedding computation and visualization (“Visual-CoT with Code Execution”) into collaborative problem-solving workflows (Chen et al., 12 Feb 2025).

6. Empirical Results and Ablation Analysis

Empirical evaluation on MathCanvas-Bench demonstrates strong domain generalization:

• On MathCanvas-Bench: BAGEL-Canvas achieves a 34.4% weighted score, compared to the base BAGEL’s 18.5% (+86% relative improvement).
• Domain-specific gains: Trigonometry (+27.1), Plane Geometry (+19.2), Solid Geometry (+12.3), Analytic Geometry (+14.1), Algebra (+11.8).
• Transfer to public benchmarks: MathVista (GPS +10.5), MathVerse (+16.2), MathVision (+8.8 overall; Analytic Geometry +22.6).

Ablation studies confirm:

• Removal of MathCanvas-Edit or -Imagen degrades performance by –2.4 to –3.6 points.
• Models fine-tuned text-only or omitting images at inference lose 2.5–3.5 points weighted accuracy.

These findings indicate not just greater accuracy, but superior alignment between the model’s problem-solving “scratchpad” and human expert practice.

7. Integration with Collaborative and Tutoring Platforms

Interactive Sketchpad exemplifies adoption of MathCanvas principles in educational contexts, harnessing programmatic diagram generation (via Python and libraries such as matplotlib, scipy) tethered to an LMM’s chain-of-thought (Chen et al., 12 Feb 2025). Visualization is selectively triggered by system prompts (“When is visualization helpful?”), with diagrams, code, and textual hints interwoven in the user’s interaction loop. The architecture supports bidirectional annotation, where users can send altered diagrams back to the model for further reasoning.

User studies highlight substantial gains in task comprehension, solution accuracy, and engagement. Tutorials alternating Interactive Sketchpad with text-only baselines yielded high scores in interface intuitiveness (4.71/5), clarity and accuracy of visuals (~4.7/5), and effectiveness of hints (4.57/5). Task-level accuracy on benchmarks such as IsoBench surpassed strong LMM baselines: Max-Flow at 100% vs 25% for GPT-4o.

A plausible implication is that tightly integrated code-driven visualization and strategic hinting, as instantiated in MathCanvas-aligned platforms, substantively enhance mathematical learning and collaborative problem-solving.

• "MathCanvas: Intrinsic Visual Chain-of-Thought for Multimodal Mathematical Reasoning" (Shi et al., 16 Oct 2025)
• "Interactive Sketchpad: A Multimodal Tutoring System for Collaborative, Visual Problem-Solving" (Chen et al., 12 Feb 2025)