Socratic-Geo: Autonomous Geometric Reasoning

• Socratic-Geo is a multi-agent framework that autonomously generates and validates synthetic geometric data to tackle MLLM limitations in problem-solving.
• It uses a closed-loop curriculum with Teacher, Solver, and Generator agents to dynamically design and expand challenging geometric tasks.
• Empirical results demonstrate state-of-the-art performance with only 2.5K problems, achieving significantly higher data efficiency than previous methods.

Socratic-Geo is a fully autonomous multi-agent framework for synthetic data generation and geometric reasoning, designed to address limitations in multimodal LLMs (MLLMs) with respect to geometric problem-solving. By dynamically coupling data synthesis with model learning using a closed-loop interplay among three specialized agents—a Teacher, a Solver, and a Generator—Socratic-Geo achieves state-of-the-art performance in geometric reasoning tasks with orders-of-magnitude greater data efficiency relative to prior methods (Jiao et al., 3 Feb 2026).

1. System Architecture

Socratic-Geo features a closed-loop curriculum engine composed of three distinct but interacting agents: Teacher (T), Solver (S), and Generator (G). The curriculum $\mathcal{C}_t$ is iteratively expanded as follows:

$$\mathcal{C}_{t+1} = \mathcal{C}_t \cup \{(I', q', a') ~|~ \exists (I, q, a^*) \in \mathcal{C}_t, S \text{ fails } k \text{ times on } (I, q), T \text{ invents and validates } (I', q', a') \}$$

• Solver (S): Given an image $I$ and question $q$, the Solver attempts solutions via policy $\pi_S$, receiving binary rewards $R \in \{0,1\}$.
• Teacher (T): Diagnoses failures of S, constructs new programmatic problems through the RePI (Reflective Problem Invention) mechanism, and employs Reflect for solvability and validity checks. Valid triplets $(I', q', a', c')$ are added to the evolving curriculum.
• Generator (G): Trains on the collection of (instruction, image) pairs accumulated from T, distilling drawing intelligence into a diffusion-based image generation model.

Interaction Workflow

$R \in \{0,1\}$1 This closed-loop design ensures that data generation is tightly coupled to the Solver’s learning needs, yielding maximally informative synthetic curricula.

2. Teacher Agent: RePI and Reflect

RePI Pipeline

• Purpose: Invents new problems based on Solver failures and modifies base Python code accordingly to create new images and textual descriptions.
• Process: From a failed instance $(I, q, a^*)$ and unsuccessful Solver attempts $\{a_S^{(i)}\}_{i=1}^k$, T:
  • Diagnoses the underlying failure $C$,
  • Modifies the base code $$\mathcal{C}_{t+1} = \mathcal{C}_t \cup \{(I', q', a') ~|~ \exists (I, q, a^*) \in \mathcal{C}_t, S \text{ fails } k \text{ times on } (I, q), T \text{ invents and validates } (I', q', a') \}$$0, and
  • Executes $$\mathcal{C}_{t+1} = \mathcal{C}_t \cup \{(I', q', a') ~|~ \exists (I, q, a^*) \in \mathcal{C}_t, S \text{ fails } k \text{ times on } (I, q), T \text{ invents and validates } (I', q', a') \}$$1 to generate $$\mathcal{C}_{t+1} = \mathcal{C}_t \cup \{(I', q', a') ~|~ \exists (I, q, a^*) \in \mathcal{C}_t, S \text{ fails } k \text{ times on } (I, q), T \text{ invents and validates } (I', q', a') \}$$2.

Reflect Validation

• Reflect: T attempts to solve $$\mathcal{C}_{t+1} = \mathcal{C}_t \cup \{(I', q', a') ~|~ \exists (I, q, a^*) \in \mathcal{C}_t, S \text{ fails } k \text{ times on } (I, q), T \text{ invents and validates } (I', q', a') \}$$3 under its own policy $$\mathcal{C}_{t+1} = \mathcal{C}_t \cup \{(I', q', a') ~|~ \exists (I, q, a^*) \in \mathcal{C}_t, S \text{ fails } k \text{ times on } (I, q), T \text{ invents and validates } (I', q', a') \}$$4, producing $$\mathcal{C}_{t+1} = \mathcal{C}_t \cup \{(I', q', a') ~|~ \exists (I, q, a^*) \in \mathcal{C}_t, S \text{ fails } k \text{ times on } (I, q), T \text{ invents and validates } (I', q', a') \}$$5. If $$\mathcal{C}_{t+1} = \mathcal{C}_t \cup \{(I', q', a') ~|~ \exists (I, q, a^*) \in \mathcal{C}_t, S \text{ fails } k \text{ times on } (I, q), T \text{ invents and validates } (I', q', a') \}$$6 matches $$\mathcal{C}_{t+1} = \mathcal{C}_t \cup \{(I', q', a') ~|~ \exists (I, q, a^*) \in \mathcal{C}_t, S \text{ fails } k \text{ times on } (I, q), T \text{ invents and validates } (I', q', a') \}$$7 and satisfies geometric consistency checks, $$\mathcal{C}_{t+1} = \mathcal{C}_t \cup \{(I', q', a') ~|~ \exists (I, q, a^*) \in \mathcal{C}_t, S \text{ fails } k \text{ times on } (I, q), T \text{ invents and validates } (I', q', a') \}$$8 is incorporated into the curriculum.

Scoring Metrics

• $$\mathcal{C}_{t+1} = \mathcal{C}_t \cup \{(I', q', a') ~|~ \exists (I, q, a^*) \in \mathcal{C}_t, S \text{ fails } k \text{ times on } (I, q), T \text{ invents and validates } (I', q', a') \}$$9
• $I$0

Pseudocode Excerpt

$R \in \{0,1\}$2

3. Solver Agent: Group Relative Policy Optimization

The Solver uses a reinforcement learning objective based on Group Relative Policy Optimization (GRPO). Rather than learning by supervised fine-tuning (SFT), S only receives binary rewards $I$1 for each attempt.

• Preference Sets: For each problem $I$2 and $I$3 attempts:
  • If $I$4, $I$5 all correct attempts, $I$6 incorrect.
  • If $I$7, $I$8 Teacher-provided reference, $I$9 all attempts.
• GRPO Loss:

$q$0

where $q$1 is the group-normalized advantage, $q$2, and $q$3 is a KL penalty coefficient.

• Failure-Guided Augmentation: Consistent Solver failures (all $q$4) prompt the Teacher to inject the reference solution as the sole positive example, directly addressing S's specific weaknesses.

4. Generator Agent: Vision Distillation

The Generator G is trained as a diffusion-based vision model on the dataset $q$5 of (drawing-instruction, image) pairs created by T.

• Distillation Loss (SFT):

$q$6

where $q$7, and $q$8 is the programmatic instruction. This distills programmatic drawing intelligence into a text-to-diagram model.

• Independence: G is trained solely on accumulated triplets, enabling standalone diagram synthesis generalizable beyond the dataset.

5. Training Regimen and Data Efficiency

Multi-Agent Training Loop

1. Initialization: The curriculum $q$9 is seeded with 108 human-verified geometric problems.
2. Stage-wise Expansion: For stages 1 to 3:
   • For each $\pi_S$0 in $\pi_S$1:
     • S attempts $\pi_S$2 solutions.
     • If all fail, T activates RePI and Reflect to generate new, verified challenges $\pi_S$3 appended to $\pi_S$4.
   • Generator collects drawing-instruction data for training.
   • S is trained via GRPO on $\pi_S$5.
   • G is trained by SFT on $\pi_S$6, accumulating across all stages.

Data Efficiency

Socratic-Geo demonstrates dramatically greater data efficiency compared to prior baselines:

• Baselines require $\pi_S$7-$\pi_S$8K synthetic samples.
• Socratic-Solver achieves state-of-the-art with only $\pi_S$9K problems ($R \in \{0,1\}$0 of baseline data).
• Curriculum grows: Stage 1 ≈ 0.4K, Stage 2 ≈ 1.0K, Stage 3 ≈ 2.5K.

6. Empirical Performance and Ablation Results

Solver Benchmarks (Mean@1%)

Model	Data (k)	MathVerse	GeomVerse	GeoQA	MathVision	MathVista	WeMath	Overall
Qwen2.5 (Zero-shot)	–	39.59	3.33	43.92	22.70	61.10	57.59	44.98
+ R-CoT (7.2k)	7.2	40.86	3.33	46.49	22.72	62.60	57.59	46.05
+ GeoReasoning	10.0	40.99	5.56	46.76	24.34	63.40	57.90	46.68
+ TrustGeoGen	10.0	41.02	4.44	46.35	22.89	61.80	58.61	46.13
Socratic-Solver (Stage 3)	2.5	45.05	6.67	49.20	26.19	63.55	61.58	49.11

Generator Benchmark (GenExam-Math)

Model	Strict (%)	Relaxed (%)
Qwen-Image (base)	0.0	18.9
Seedream-4.0 (closed-source)	2.6	39.8
Gemini-2.5-Flash-Image	0.7	43.1
Socratic-Generator-Image	6.0	42.4

Ablation Studies

• Removing Reflect reduces Stage 1 performance (on ~0.4K verified samples) from 40.33% to 37.09% (on 1.3K unverified).
• Excluding instruction rewriting for G produces relaxed GenExam-Math accuracy of 20.1% versus 42.4% with rewriting.
• Reinforcement learning outperforms SFT on identical 2.5K data: 49.11% (RL) vs 47.37% (SFT).
• Teacher size less critical than information access; a 3B-parameter T attains 48.47% (vs. 49.11% for 235B).

7. Significance, Limitations, and Prospective Directions

Key Insights

• Coupling synthetic generation directly to Solver failures yields curricula targeting key weaknesses, maximizing sample efficiency.
• Use of programmatic control (RePI, Reflect) enforces problem and solution validity, facilitating reliable training signals.
• Code-driven instruction distillation empowers G to learn diagrammatic generation from scratch, without access to external datasets.

Current Limitations

• Restriction to Euclidean geometry; generalization to algebraic domains or 3D geometry is not presently realized.
• High-quality Teacher model requirement for effective code modification may limit deployment in resource-constrained scenarios.

Potential Extensions

• Application of the framework to chart reasoning, multimodal coding, physics diagrams, or chemistry reaction mechanisms.
• Advancing meta-learning paradigms for the Teacher to adapt invention strategies autonomously.
• Incorporation of human-in-the-loop feedback for expert-validated problem generation and curriculum refinement.

(Jiao et al., 3 Feb 2026)