Multiscale Logical Evaluation (MLE)

• Multiscale Logical Evaluation (MLE) is a framework that dissects logical reasoning into distinct scales—local vs global, micro vs macro—to better diagnose and optimize reasoning quality.
• MLE frameworks like SATQuest, ME², LogicScore, and MRNet use controlled decomposition and diverse verification techniques to reveal failure modes that are hidden by flat accuracy metrics.
• These methods integrate solver-backed checks, DAG abstractions, and multiresolution analysis to provide actionable insights, even as complexity increases and minimality challenges arise.

Searching arXiv for the cited works to ground the article in current metadata and confirm relevant papers. Multiscale Logical Evaluation (MLE) denotes a family of evaluation frameworks in which logical quality is analyzed across more than one scale of structure rather than reduced to a single end-task correctness signal. In current arXiv usage, the term has been instantiated in several technically distinct but conceptually aligned ways: SATQuest operationalizes MLE for LLMs through a grid over instance scale, SAT-based problem type, and question format; the ME$^2$ framework treats MLE as macro- versus micro-level reasoning assessment under efficiency and effectiveness criteria; LogicScore applies MLE to attributed question answering through local, intermediate, and global logical checks; and MRNet applies multiscale logical reasoning to abstract visual analogy by combining multiple image resolutions with grid-structured relational operators (Zhao et al., 31 Aug 2025, Zhang et al., 9 Feb 2026, Yan et al., 21 Jan 2026, Benny et al., 2020).

1. Conceptual schema and scope

Across these formulations, MLE is not a single metric, benchmark, or model. It is better understood as an evaluation schema in which logical competence is decomposed into structurally distinct axes, and reasoning quality is then measured, diagnosed, or optimized along those axes. This suggests that “multiscale” refers not only to input size, but also to representational level: local versus global proof structure, shallow versus deep combinatorial difficulty, or fine versus coarse perceptual evidence.

Instantiation	Evaluated object	Core multiscale dimensions
SATQuest (Zhao et al., 31 Aug 2025)	SAT-based LLM reasoning outputs	$S \times T \times F$: instance scales, problem types, formats
ME$^2$/TRM (Zhang et al., 9 Feb 2026)	Reasoning traces	macro/micro $\times$ efficiency/effectiveness
LogicScore (Yan et al., 21 Jan 2026)	Long-form attributed QA	local/intermediate/global logic
MRNet (Benny et al., 2020)	RPM-style visual reasoning	high/middle/low image scales

The common methodological pattern is controlled decomposition. SATQuest varies combinatorial scale while holding logical semantics fixed. ME$^2$ maps reasoning traces into DAGs so that macro organization and micro step quality can be judged separately. LogicScore converts long-form answers into Horn-rule proof objects and verifies whether a minimal chain from question to answer exists. MRNet separates perceptual evidence by resolution and then enforces logical consistency across rows and columns. Taken together, these works indicate that MLE functions as a general design principle for exposing failure modes that remain hidden under flat accuracy measures.

2. SAT-based MLE: task-space decomposition in SATQuest

SATQuest defines perhaps the most explicit MLE grid. It starts from CNF formulas over propositional variables $X=\{x_1,\ldots,x_n\}$, with

$$\phi = \bigwedge_{i=1}^m C_i,\qquad C_i = \bigvee_{j=1}^{k_i} \ell_{ij},$$

where each literal $\ell_{ij}\in\{x,\neg x\}$ for some $x\in X$, and satisfiability is defined by the existence of an assignment $\sigma:X\to\{0,1\}$ such that every clause evaluates to true. On top of this substrate, SATQuest defines five tasks: SAT Decision Problem (SATDP), SAT Search Problem (SATSP), Maximum Satisfiability (MaxSAT), Minimal Correction Subset (MCS), and Minimal Unsatisfiable Subset (MUS) (Zhao et al., 31 Aug 2025).

Its multiscale structure is the grid $S \times T \times F$0. The scale axis $S \times T \times F$1 comprises values of $S \times T \times F$2, $S \times T \times F$3, the clause-to-variable ratio $S \times T \times F$4, and derived solver-complexity statistics such as decisions, conflicts, and propagations. The type axis $S \times T \times F$5 is the five-task set $S \times T \times F$6. The format axis $S \times T \times F$7 contains four logically equivalent encodings: mathematical notation, DIMACS CNF machine format, narrative OR semantics, and narrative AND semantics. This design makes it possible to vary combinatorial difficulty, task objective, and linguistic surface form independently.

The framework generates paired UNSAT and SAT CNFs using a stochastic pipeline adapted from the NeuroSAT generator. Clause widths are sampled with unit clauses controlled by $S \times T \times F$8 and longer clauses by a geometric distribution with parameter $S \times T \times F$9; an UNSAT instance is regenerated until solver-checked unsatisfiability holds, and a matched SAT instance is created by flipping literal polarities until satisfiability holds. The paper emphasizes that paired SAT/UNSAT instances have near-identical structure and literal counts, which reduces shortcut exploitation and memorization risk. Evaluation data comprise 140 CNF pairs for $^2$0 with $^2$1, and reinforcement fine-tuning data comprise 3,000 pairs for $^2$2 and $^2$3 in 0.1 steps.

Verification is fully solver-backed. PySAT is used with Glucose 4.1 for SATDP and SATSP, RC2 for MaxSAT, LBX for MCS, and MUSX for MUS. Answers are encoded as binary strings: 1-bit outputs for SATDP, $^2$4-bit assignments for SATSP and MaxSAT, and $^2$5-bit clause masks for MCS and MUS. The verifier $^2$6 checks exact satisfiability, optimality, or minimality, depending on task. Reported evaluation metrics are accuracy, response length, and format correctness.

The empirical picture is strongly scale-sensitive. Overall accuracy is reported as 0.56 for o3-mini, 0.42 for DeepSeek-R1, 0.40 for QwQ-32B, 0.39 for DeepSeek-R1-Distill-Qwen-32B, 0.38 for GPT-4.1, and 0.36 for DeepSeek-V3-0324, while smaller vanilla models such as Qwen2.5-7B-Instruct are below 0.1. SATDP and SATSP are comparatively easier, MaxSAT is intermediate, and MCS and MUS are hardest. Mathematical notation is the easiest format; DIMACS lags despite being structurally compact; narrative OR and narrative AND are hardest. As instance complexity rises, accuracy declines, response length generally increases, and hallucination behaviors emerge.

SATQuest also uses MLE as a training substrate. With GRPO implemented via TRL, Qwen2.5-7B-Instruct is fine-tuned on SATSP in mathematical notation, MaxSAT in mathematical notation, and SATDP in narrative OR semantics. Rewards comprise a primary correctness term $^2$7 if $^2$8 and $^2$9 otherwise, plus two shaping rewards of 0.05 each for tag structure and formatting. The reported outcome is that fine-tuning improves targeted tasks and generalizes to larger instances of the same type and format, but cross-format transfer remains limited; training on MaxSAT helps SATSP, whereas training on SATSP does not comparably improve MaxSAT.

3. Trace-space MLE: macro/micro reasoning assessment with ME$\times$0

The paper "Characterizing, Evaluating, and Optimizing Complex Reasoning" formalizes MLE through the ME$\times$1 principle, which divides reasoning quality by where it manifests and what it consists in. The first axis is macro versus micro: global structural organization versus local step properties. The second is efficiency versus effectiveness: resource discipline versus logical adequacy. The resulting four dimensions are Macro-Efficiency, Macro-Effectiveness, Micro-Efficiency, and Micro-Effectiveness (Zhang et al., 9 Feb 2026).

Macro-Efficiency concerns global structural discipline, including unnecessary branching, reflection, restarts, and reopened branches. Macro-Effectiveness concerns coherence and goal alignment across the whole trace. Micro-Efficiency concerns local concision and the avoidance of filler, hedging, or looping paraphrase. Micro-Effectiveness concerns local validity, computational soundness, and internal consistency. This framework treats reasoning quality as a structured property of traces, not merely as answer correctness.

To operationalize these dimensions, reasoning traces are split into steps and represented as DAGs $\times$2. Nodes correspond to atomic steps or compressed super-nodes, and edges encode semantic dependency while preserving causal order. The construction procedure consists of step partitioning, incremental parent selection from an attachment pool that includes main-branch nodes and representative leaves from other branches, and chain compression. Two abstractions are then derived: a macro abstraction $\times$3 that linearizes super-nodes with summaries and structural metadata, and a dominant path $\times$4 whose concatenated raw texts define the micro abstraction.

Evaluation is pairwise. Given two traces, an evaluator produces four judgments $\times$5, one for each ME$\times$6 dimension, together with short rationales. Order is reversed and evaluation is repeated to reduce position bias; only consistent non-tied labels are retained. A final preference is then aggregated from the four dimension-wise judgments. Reward modeling follows the Bradley–Terry formulation,

$\times$7

with corresponding BT loss over preference pairs.

This protocol is used to build the TRM-Preference dataset from WebInstruct-verified. The paper reports 64K sampled prompts, 103K training pairs, and 1.5K validation pairs, using only verified-correct traces so that reasoning quality is decoupled from final-answer correctness. The resulting Thinking Reward Model (TRM) is a Llama-3.1-8B-Instruct model with a scalar value head trained via trl RewardTrainer. Reported validation accuracy is 88.6%, compared with 46.3% for Qwen2.5-Math-PRM-7B, 62.5% for ReasonFlux-PRM-7B, and 78.6% for prompt-only judging on raw text.

MLE here is not only evaluative but also optimization-oriented. At test time, Best-of-$\times$8 selection uses the TRM score $\times$9 to choose the best trace among sampled candidates; reported gains reach 19.3% on AIME24 with Qwen3-8B, from 44.7% at $^2$0 to 64.0% at $^2$1. During RL training, the paper combines binary verifiable reward $^2$2 with a thinking reward $^2$3 through gated shaping,

$^2$4

with best aggregate performance reported at $^2$5. Reported training gains reach 3.9% over verifier-only RL in STEM averages. A central implication is that MLE can supply graded optimization signal even when end-task correctness has already saturated among successful trajectories.

4. Proof-oriented MLE in attributed question answering: LogicScore

LogicScore applies MLE to attributed question answering by shifting evaluation away from isolated claim verification and toward global logical integrity. The paper characterizes the dominant failure of earlier evaluators as attribution myopia: metrics such as AutoAIS and FACTSCORE can verify that individual statements are supported by cited passages while missing broken bridges, circular chains, redundancy, or ambiguity in the answer as a whole (Yan et al., 21 Jan 2026).

Its logical substrate is a Horn-rule formalization. A reasoning path is written as

$^2$6

where each $^2$7 is an atomic proposition derived from the long-form answer and $^2$8 is the short answer. More generally, the framework uses rules of the form

$^2$9

Given a question $X=\{x_1,\ldots,x_n\}$0, short answer $X=\{x_1,\ldots,x_n\}$1, and long-form answer $X=\{x_1,\ldots,x_n\}$2, an LLM transforms $X=\{x_1,\ldots,x_n\}$3 into atomic propositions and a Horn clause $X=\{x_1,\ldots,x_n\}$4, extracts triples from each proposition, and runs backward chaining from the answer entity to the question entity.

This yields three MLE metrics. Completeness is binary:

$X=\{x_1,\ldots,x_n\}$5

where $X=\{x_1,\ldots,x_n\}$6 is the minimal sufficient set recovered by backward search. Conciseness is

$X=\{x_1,\ldots,x_n\}$7

when a valid chain exists and 0 otherwise. Determinateness is

$X=\{x_1,\ldots,x_n\}$8

where $X=\{x_1,\ldots,x_n\}$9 is re-inferred from $$\phi = \bigwedge_{i=1}^m C_i,\qquad C_i = \bigvee_{j=1}^{k_i} \ell_{ij},$$0 in a closed-world setting. The three metrics correspond, respectively, to logical connectivity, non-redundancy, and unambiguous entailment.

The search procedure builds a proposition graph in which nodes are propositions and edges connect propositions sharing entities after triple extraction. Backward chaining begins from propositions containing the gold answer entity, traverses through shared bridge entities, and succeeds only if a path reaches the question entity. With naive matching, worst-case complexity is $$\phi = \bigwedge_{i=1}^m C_i,\qquad C_i = \bigvee_{j=1}^{k_i} \ell_{ij},$$1 in the number of propositions, while entity-indexed lookup reduces candidate retrieval to near $$\phi = \bigwedge_{i=1}^m C_i,\qquad C_i = \bigvee_{j=1}^{k_i} \ell_{ij},$$2 per hop.

Empirically, LogicScore evaluates more than 20 LLMs on HotpotQA, MusiQue, and 2WikiMultiHopQA. The headline result is a large attribution-versus-logic gap: the abstract reports that Gemini-3 Pro achieves 92.85% attribution precision but only 35.11% Conciseness. On MusiQue, strong models still show only moderate Completeness and low Conciseness; for example, Gemini-3-Pro is reported at 59.21% Completeness, and GPT-o3 at 37.30% Conciseness. The paper also reports a scaling paradox for Qwen3: larger models show higher Determinateness but often lower Conciseness. Human alignment is reported as 90.84% Pearson correlation for Conciseness, 94.39% Jaccard for Completeness, and 92.01% Jaccard for Determinateness, with logic transformation accuracy at 96.75%.

The framework identifies three recurrent pathologies: Circular, Broken, and Deviated chains. In the “Top Gun theme” case study, backward verification fails because no proposition connects Steve Stevens to Warner Bros. Records, so Completeness and Conciseness are both 0; Determinateness is also 0 because re-inference from the long-form answer yields “James Conkling” rather than “Warner Music Group.” This use of MLE is thus explicitly proof-theoretic: answers are judged not only for support, but for whether their internal proposition graph actually entails the claimed conclusion.

5. Multiresolution logical evaluation in abstract visual reasoning

In "Scale-Localized Abstract Reasoning," MLE is instantiated in a perceptual setting rather than a textual one. The task is Raven-style abstract relational reasoning over a $$\phi = \bigwedge_{i=1}^m C_i,\qquad C_i = \bigvee_{j=1}^{k_i} \ell_{ij},$$3 image grid with one missing panel. The paper studies two benchmarks: PGM, with 1.2M training examples, 20K validation, and 200K test; and RAVEN, with 42K training, 14K validation, and 14K test examples across seven structural configurations (Benny et al., 2020).

The key claim is that different relational rules are best captured at different resolutions. Spatial and location-sensitive rules require high-resolution evidence, whereas semantic or global-gestalt rules are better captured at lower resolution. MRNet therefore encodes each image at three scales:

$$\phi = \bigwedge_{i=1}^m C_i,\qquad C_i = \bigvee_{j=1}^{k_i} \ell_{ij},$$4

with feature dimensions $$\phi = \bigwedge_{i=1}^m C_i,\qquad C_i = \bigvee_{j=1}^{k_i} \ell_{ij},$$5, $$\phi = \bigwedge_{i=1}^m C_i,\qquad C_i = \bigvee_{j=1}^{k_i} \ell_{ij},$$6, and $$\phi = \bigwedge_{i=1}^m C_i,\qquad C_i = \bigvee_{j=1}^{k_i} \ell_{ij},$$7, respectively.

At each scale, row and column triplets are processed through a shared relation network $$\phi = \bigwedge_{i=1}^m C_i,\qquad C_i = \bigvee_{j=1}^{k_i} \ell_{ij},$$8, yielding row features $$\phi = \bigwedge_{i=1}^m C_i,\qquad C_i = \bigvee_{j=1}^{k_i} \ell_{ij},$$9 and column features $\ell_{ij}\in\{x,\neg x\}$0. These are pooled with the paper’s triple-distance operator

$\ell_{ij}\in\{x,\neg x\}$1

producing per-scale features $\ell_{ij}\in\{x,\neg x\}$2, bottlenecked to vectors $\ell_{ij}\in\{x,\neg x\}$3 and concatenated into a multiscale representation $\ell_{ij}\in\{x,\neg x\}$4. The model also includes per-scale auxiliary heads and a performance-proportional loss

$\ell_{ij}\in\{x,\neg x\}$5

where the weights $\ell_{ij}\in\{x,\neg x\}$6 are a softmax over each head’s correctness probability on the current sample. This induces scale specialization.

The reported ablations strongly support the multiscale design. Replacing DIST3 with SUM3 reduces single-choice accuracy from 93.4% to 83.2% on PGM, from 92.6% to 85.3% on PGM_meta, from 86.8% to 79.5% on RAVEN-FAIR, and from 84.0% to 78.2% on RAVEN. Removing the auxiliary multi-head loss also sharply degrades performance. Single-scale variants reveal specialization: high resolution excels on location rules such as L-R, U-D, and O-IC; low resolution excels on Center, $\ell_{ij}\in\{x,\neg x\}$7, and $\ell_{ij}\in\{x,\neg x\}$8 configurations; and the middle scale performs best on O-IG.

The paper also gives MLE a benchmark-design dimension. It argues that RAVEN’s original negative-choice construction is exploitable by context-blind heuristics, reporting 80.17% blind accuracy. To address this, it proposes RAVEN-FAIR, which iteratively constructs negatives by modifying attributes while ensuring that each candidate does not satisfy the context rules. RAVEN-FAIR reduces blind accuracy to 17.24%, near the 12.5% random baseline for eight-way multiple choice. In this visual setting, MLE therefore includes both multiresolution reasoning and fairness-aware evaluation design.

6. Cross-framework themes, limitations, and terminological ambiguity

Several common themes recur across these MLE instantiations. First, logical competence is strongly scale-dependent. In SATQuest, accuracy deteriorates as solver decisions rise and is especially fragile on MaxSAT, MCS, and MUS. In LogicScore, Conciseness, Completeness, and Determinateness decline as hop depth increases from 2 to 4. In MRNet, different resolutions specialize on different rule families. In ME$\ell_{ij}\in\{x,\neg x\}$9, macro organization and micro validity make separable contributions to reasoning quality (Zhao et al., 31 Aug 2025, Yan et al., 21 Jan 2026, Benny et al., 2020, Zhang et al., 9 Feb 2026).

Second, MLE frameworks differ in the source of supervision. SATQuest uses exact solver-backed verification over binary string outputs. LogicScore uses proof-search and re-inference over LLM-transformed Horn clauses. ME$x\in X$0 uses robust pairwise LLM-as-judge preferences over DAG abstractions and then distills them into a reward model. MRNet relies on supervised answer prediction but evaluates which scale-specific relational operators capture the relevant logic. This suggests that MLE is compatible with exact verifiers, structured symbolic search, preference models, and standard discriminative training, provided that reasoning is decomposed into inspectable scales.

Third, each formulation has clear limitations. SATQuest reports limited cross-format generalization and persistent difficulty on minimality tasks. ME$x\in X$1 depends on LLM-based DAG construction and judging, with position bias and structural recovery errors as acknowledged risks. LogicScore depends on logic transformation quality and on the adequacy of definite-clause assumptions for real answers. MRNet assumes fixed grid structure and still struggles on held-out extrapolative regimes. A plausible implication is that MLE exposes failure modes more effectively than flat metrics, but does not by itself solve representation, transfer, or judge-reliability problems.

Finally, the acronym “MLE” is terminologically ambiguous in the broader literature. In multiscale diffusion modeling, for example, MLE denotes Maximum Likelihood Estimation rather than Multiscale Logical Evaluation, and the paper "Coarse-grained modeling of multiscale diffusions: the p-variation estimates" discusses the consistency of maximum-likelihood estimators under averaging and homogenization, together with a $x\in X$2-variation estimator for diffusion coefficients (Papavasiliou, 2010). This usage is unrelated to logical evaluation, but it is well established and can create ambiguity outside contexts where “Multiscale Logical Evaluation” is explicitly defined.

Taken together, contemporary usages of MLE describe a research program rather than a single formal object: logical reasoning is treated as a multilevel phenomenon whose quality depends on scale, structure, representation, and verification regime. SATQuest emphasizes combinatorial and format-sensitive logical robustness; ME$x\in X$3 emphasizes structured trace quality and optimization; LogicScore emphasizes proof connectivity, minimality, and entailment in long-form answers; and MRNet emphasizes resolution-specific relational abstraction. The shared conclusion is that evaluating reasoning at only one level—final answer, local attribution, flat trace text, or single resolution—systematically misses important logical behavior.