UltraLogic: Scalable LLM Logical Reasoning

• UltraLogic is a framework that decouples logical task structure from natural language through a code-based solving paradigm, ensuring verifiable data synthesis.
• It uses a novel Bipolar Float Reward scheme to provide clear feedback by differentiating perfect solutions from flawed outputs with a 'Reward Cliff'.
• The system adapts training difficulty through curriculum learning, validated by empirical benchmarks that show scalable improvements in LLM reasoning.

UltraLogic is a framework developed to advance general-purpose, multi-step logical reasoning in LLMs via large-scale, high-quality data synthesis and novel reward design. It achieves this by decoupling the logical structure of reasoning tasks from their natural language expressions, automating data creation across a broad taxonomy of logical domains, and employing a Bipolar Float Reward (BFR) scheme specifically tailored to address critical limitations in conventional reinforcement learning approaches. The methodology, calibration protocols, and empirical benchmarks establish UltraLogic as a significant step in efficient and scalable training of reasoning-oriented models (Liu et al., 6 Jan 2026).

1. Code-based Problem Decomposition and Data Synthesis

The central methodological innovation in UltraLogic is the code-based solving paradigm, which separates the logical core of a task from its linguistic rendering. Each task is encoded through two main functions:

• Input generation: A function $$\text{input}:\{1,\ldots,10\}\times\{\texttt{en},\texttt{zh}\}\rightarrow \mathcal{P}\times\{\text{text templates}\}$$ that samples a set of logical parameters $p\in\mathcal{P}$ and selects a natural language template according to requested difficulty and language.
• Solution function:

$$\text{solution}:\mathcal{P}\times\{\texttt{en},\texttt{zh}\}\rightarrow A$$ that deterministically computes the ground-truth answer for the exact parameters $p$.

This separation ensures all generated data points are inherently verifiable by re-executing the solution code. The process relies on a semi-automated, multi-step pipeline:

1. Template Deconstruction: Domain experts and LLMs identify the logical skeleton and parameter slots, then formalize the base template.
2. Template Expansion: The base form is diversified into narrative variants while keeping slot structure intact, followed by manual refinement into a bilingual template library.
3. Code Generation and Verification: LLMs generate the corresponding Python functions for input and solution; annotators debug and assure exact alignment.

The complete codebase allows for unlimited fresh sample generation, controlled precisely by difficulty parameters.

2. Task Taxonomy and Difficulty Calibration

UltraLogic’s original dataset encompasses approximately 200 task types, each characterized along three orthogonal axes:

• Task Domain (e.g., Symbolic Manipulation, Spatial Pathfinding, Planning/Scheduling, Classic Games)
• Core Reasoning Ability (e.g., Constraint Satisfaction, Algorithmic Thinking, Information Extraction, Instruction Following)
• Source of Difficulty (e.g., Large Search Space, Complex Rules, Tedious Solution Steps, Computational Complexity, Intrinsic LLM Weakness)

Every task is labeled by the triplet $(\text{Domain},\,\text{Ability},\,\text{Source})$, supporting combinatorial diversity in reasoning challenges.

Difficulty is stratified into 10 discrete levels, calibrated against flagship LLMs. The calibration protocol is defined by target success rates: $$P_{\text{target}}(d) = \begin{cases} \approx 1.00, & d=1\ 0.70, & d=3\ 0.50, & d=5 \ 0.30, & d=7\ 0.00, & d=10\ \end{cases}$$ Internal task parameters $\theta_d$ are optimized through closed-loop search (e.g., proportional–integral or binary search techniques) to ensure that observed model accuracy matches targets, yielding an objective, reproducible 1–10 ladder extendable to future models by recalibration.

3. Bipolar Float Reward (BFR) and Reinforcement Design

Conventional RLVR with binary rewards

$$R_i\in \{0,1\},\quad R_i = 1 \iff \text{correct}(p_i)$$

suffers from the Non-negative Reward Trap: partial but imperfect solutions all receive reward zero, causing insufficient gradient signal and convergence to suboptimal behaviors.

UltraLogic introduces BFR, which enforces a bipolar separation between perfect and flawed responses: $$R_{\mathrm{BFR}}(S) = \begin{cases} 1.0, & S = 1 \ S-1.0, & 0 \leq S < 1 \end{cases}$$ where $S$ is a task-specific, graded correctness metric (e.g., exact accuracy, F1 score, similarity, or absolute error rate). BFR guarantees that only perfect answers receive +1, while all others receive a negative reward proportional to their deficit, producing a “Reward Cliff” that propels the policy toward globally correct outputs.

A small formatting bonus of +0.1 is implemented in all reward schemes to distinguish logical from mere presentation errors. No other thresholds are used; grading is fully linear.

4. Adaptive Difficulty Matching

UltraLogic exploits curriculum learning principles by monitoring the zero-shot success rate $p\in\mathcal{P}$0 of the current model at each difficulty. The optimal training signal arises from instances where $p\in\mathcal{P}$1 is close to 0.5, echoing the psychometric “Zone of Proximal Development.” The training algorithm selects difficulty via: $p\in\mathcal{P}$2 and samples difficulties adaptively: $p\in\mathcal{P}$3 with $p\in\mathcal{P}$4 to maintain a focus on the optimal learning window and avoid both trivial and infeasible regimes.

5. Empirical Evaluation and Ablations

Empirical validation was conducted using Qwen3-8B and Qwen3-14B models, optimized with Group Relative Policy Optimization (GRPO). Benchmarks included AIME 2024/25, HMMT 2025, BBH, BBEH, and ARC-AGI. Training consisted of 50 tasks, up to 10 k samples each, over 2 epochs.

Difficulty matching ablation:

Qwen3-8B showed best performance gains when trained on “Easy” (levels 1–4); Qwen3-14B on “Medium/Hard” (levels 4–10). This outcome substantiates the value of fine-grained, model-targeted curricula.

Reward mechanism ablation:

Among binary, graded float, and BFR, the latter yielded the strongest improvement—particularly on logic-intensive benchmarks and with smoother convergence.

Reward Scheme	AIME24	HMMT25	BBH
Binary	81.7	52.3	90.2
Graded Float	76.9	53.0	90.4
Bipolar Float	82.6	56.6	91.1

Task diversity ablation:

Varying dataset scale with fixed task diversity (10k vs. 200k samples per task) induced diminishing returns beyond 50k. In contrast, adding new task types increased performance linearly, establishing task diversity (not data volume) as the primary contributor to reasoning capacity.

6. Insights, Limitations, and Future Directions

UltraLogic demonstrates that general-purpose reasoning in LLMs is predominantly enhanced via exposure to a wide spectrum of logical problems. This exposure promotes acquisition of meta-reasoning heuristics, such as constraint tracking and backtracking, which models do not acquire from narrow task distributions.

Key limitations observed include the continued need for human experts in template construction and code debugging—RLVR remains brittle to subtle logic bugs in even a few tasks. The BFR regime, while robust, is heuristic in its thresholding between perfect and imperfect solutions; exploration of meta-optimized or smoother reward cliffs is cited as a future research direction. The current discrete difficulty stratification may require extension to a broader or continuous ladder as model capabilities increase.

In summary, UltraLogic provides (i) a scalable code-based pipeline for reasoning data; (ii) an objective difficulty regime with calibrated, reproducible standards; (iii) a curriculum-optimized, model-adaptive training loop; and (iv) a principled reward system to overcome reward sparsity, establishing new best practices for LLM reasoning training (Liu et al., 6 Jan 2026).