General365: Benchmarking General Reasoning in Large Language Models Across Diverse and Challenging Tasks

Abstract: Contemporary LLMs have demonstrated remarkable reasoning capabilities, particularly in specialized domains like mathematics and physics. However, their ability to generalize these reasoning skills to more general and broader contexts--often termed general reasoning--remains under-explored. Unlike domain-specific reasoning, general reasoning relies less on expert knowledge but still presents formidable reasoning challenges, such as complex constraints, nested logical branches, and semantic interference. To address this gap, we introduce General365, a benchmark specifically designed to assess general reasoning in LLMs. By restricting background knowledge to a K-12 level, General365 explicitly decouples reasoning from specialized expertise. The benchmark comprises 365 seed problems and 1,095 variant problems across eight categories, ensuring both high difficulty and diversity. Evaluations across 26 leading LLMs reveal that even the top-performing model achieves only 62.8% accuracy, in stark contrast to the near-perfect performances of LLMs in math and physics benchmarks. These results suggest that the reasoning abilities of current LLMs are heavily domain-dependent, leaving significant room for improvement in broader applications. We envision General365 as a catalyst for advancing LLM reasoning beyond domain-specific tasks toward robust, general-purpose real-world scenarios. Code, Dataset, and Leaderboard: https://general365.github.io

• The paper introduces General365, a novel benchmark that rigorously tests LLM general reasoning using diverse, multi-step, and composite challenges based on a K-12 curriculum.
• It employs a hybrid scoring approach combining rule-based and GPT-4.1 methods to achieve a validated scoring precision of 99.6% across complex answer formats.
• Empirical results reveal that top models like Gemini-3-Pro peak at only 62.8% accuracy, highlighting the limitations of current LLM architectures in handling general reasoning tasks.

General365: A Rigorous Benchmark for General Reasoning in LLMs

Motivation and Benchmark Design

General365 addresses a central gap in LLM evaluation: the lack of robust, high-difficulty benchmarks targeting general reasoning—reasoning decoupled from reliance on domain expertise or specialized factual recall. Whereas prior benchmarks (e.g., MATH, Physunibench, BBH, BBEH) principally assess problem-solving in specialized domains or rapidly saturate in difficulty, General365 restricts the required knowledge base to the K-12 curriculum, focusing the cognitive load on reasoning across complex constraints, multi-step logic, semantic interference, and rigorous abstraction.

The construction pipeline for General365 integrates human-generated seed problems, model-driven expansion, and systematic manual quality control to ensure high diversity and minimal redundancy. Each of the 365 hand-crafted seed problems was reviewed for originality and alignment to at least one of eight reasoning categories, followed by expansion with LLMs and human-in-the-loop filtering, yielding 1,460 challenging instances. This pipeline, which is visualized below, guarantees coverage of a broad spectrum of logical structures and cognitive challenge types.

Figure 1: The General365 construction pipeline employs seeded manual creation, LLM-based expansion, and rigorous manual review for diversity and quality.

General365’s taxonomy comprises eight categories: Complex Constraints, Branching Enumeration, Spatial/Temporal Reasoning, Recursive Backtracking, Semantic Interference, Implicit Information Reasoning, Optimal Strategy, and Probability/Uncertainty. This categorization was established to systematically probe distinct reasoning primitives, including state tracking under multi-step logical predicates, search over complex solution spaces, recursive hypothesis formation, and utility-based optimization.

Figure 2: Distribution of General365 challenge categories, reflecting balanced coverage across reasoning dimensions.

Notably, nearly 70% of problems are annotated with multiple challenge labels, leading to a high incidence of composite tasks that stress simultaneous mastery of several reasoning modes.

Scoring Methodologies and Dataset Integrity

Evaluative reliability is fundamental to the benchmark. General365 employs a hybrid grading algorithm that dynamically switches between rule-based and model-based scoring based on answer format to maximize both coverage and accuracy. This approach achieves a manually validated scoring precision of 99.6%, robust even for open-ended or complex answers.

Problems predominantly require numerical, select, or text answers, each demanding different grading strategies. Numerical problems utilize canonical boxed LaTeX answers with precision constraints, verified by Math-Verify tools. Non-numeric answers are adjudicated using GPT-4.1-based criteria. The public leaderboard and dataset are accessible for reproducibility.

Empirical Evaluation and Model Performance

Extensive evaluation of 26 advanced LLMs (spanning proprietary and open-source, “reasoning” and “chat” architectures) was undertaken to expose the limits of present-day reasoning. Noteworthy results include:

• No model reaches 65% accuracy: Gemini-3-Pro, the top-performing model, achieves only 62.8% accuracy, well below the threshold at which trivial or exploitable patterns might explain performance.

  Figure 3: Gemini-3-Pro leads all tested LLMs with 62.8% accuracy on General365; most models score well below the 60% 'passing' mark.

• Open-source advances are significant: GLM-5-Thinking achieves 59.9%—within 3% of Gemini-3-Pro—narrowing the historical gap with proprietary systems.
• Non-reasoning models occasionally outperform specialized models: E.g., Qwen3-Max-Instruct surpasses Gemini-2.5-Flash.
• "Semantic Interference" and "Optimal Strategy" are consistent bottlenecks: Across models, these categories yield accuracies ~10% below the aggregate, revealing LLM vulnerability to problems requiring long-term planning or those explicitly designed to elicit incorrect, intuitive responses.

Reasoning Efficiency and Output Analysis

An important secondary metric is reasoning efficiency—the ratio of solution accuracy to average output length. Longer outputs reflect either inefficient reasoning or increased cognitive demands. Among top-tier models:

• Most require 25k–30k output tokens for adequate performance.
• Gemini-3-Pro is an outlier: it achieves SOTA accuracy (62.8%) with just 14k tokens on average, establishing a new standard for output efficiency.

  Figure 4: Gemini-3-Pro demonstrates superior accuracy with markedly fewer tokens, indicating efficient reasoning when compared to other frontier LLMs.

Furthermore, when compared to other benchmarks (BBH, BBEH), General365 compels significantly longer outputs yet produces lower accuracy, underscoring its higher cognitive complexity and lack of superficial shortcuts.

Figure 5: Top models produce much longer outputs on General365 than on BBH/BBEH, reinforcing the strict reasoning demands imposed by the benchmark.

Quantifying Diversity and Logical Non-redundancy

General365’s semantic and logical diversity is both visually and quantitatively demonstrated. Visualizations of t-SNE projected embeddings show that its instances are distributed broadly and evenly, in stark contrast with BBH/BBEH benchmarks, which reveal tight clusters and significant "local collapse"—a proxy for semantic redundancy.

Figure 6: General365’s t-SNE embeddings reveal broad, uniform distribution, in contrast to the clustered structure of BBH and BBEH, evidencing superior semantic diversity.

Logical redundancy, as measured by pairwise reasoning similarity scored by expert LLMs, is markedly lower in General365 (left-skewed, $\mu=2.16$), while BBH/BBEH are dominated by scores of 5 (structural equivalence).

Figure 7: General365 has a left-skewed similarity distribution (i.e., low logical redundancy), in contrast to the high redundancy (right-skew) of BBH/BBEH.

Implications and Future Directions

General365 exposes the pronounced domain-dependence of present-day LLMs. The substantial gap in accuracy between specialized benchmarks (where models approach or exceed 90%) and General365 (best SOTA at 62.8%) demonstrates that current architectures, despite advances in scaling and training, remain brittle at general reasoning scenarios requiring abstraction, adversarial thinking, or composite cognitive protocols.

From a practical perspective, this delineates the boundary at which LLMs can be reliably deployed for reasoning-intensive, everyday real-world tasks, and highlights the inadequacy of continued scaling or superficial prompt engineering alone to address the observed deficits. The superior efficiency profile of Gemini-3-Pro also suggests future research should couple reasoning capability with reduced inference costs.

Theoretically, future directions likely include the integration of explicit planning, memory, or self-correction mechanisms, as well as architectures that further disentangle knowledge retrieval from logical manipulation.

Conclusion

General365 establishes a stringent, well-validated benchmark for general reasoning in LLMs, revealing significant limitations in state-of-the-art models. The benchmark’s meticulous construction, diversity audits, and robust scoring methodology make it a valuable asset both for diagnostic model evaluation and for driving progress in unified reasoning architectures. "General365: Benchmarking General Reasoning in LLMs Across Diverse and Challenging Tasks" (2604.11778) will serve as a foundation for future work seeking to develop robust, real-world-ready LLMs.