CheeseBench: Evaluating Large Language Models on Rodent Behavioral Neuroscience Paradigms

Abstract: We introduce CheeseBench, a benchmark that evaluates LLMs on nine classical behavioral neuroscience paradigms (Morris water maze, Barnes maze, T-maze, radial arm maze, star maze, operant chamber, shuttle box, conditioned place preference, and delayed non-match to sample), spanning six cognitive dimensions. Each task is grounded in peer-reviewed rodent protocols with approximate animal baselines. The agent receives a unified system prompt with no task-specific instructions and must discover goals purely from ASCII text observations and reward signals, much like a rodent placed into an unfamiliar apparatus. We evaluate six open-weight LLMs (3B to 72B parameters) on text-based ASCII renderings and compare against both a random baseline and a graph-based reinforcement learning agent. Our best model (Qwen2.5-VL-7B) reaches 52.6% average success on ASCII input, compared to 32.1% for random agents and 78.9% for approximate rodent baselines. We find that (1) scaling beyond 7B yields diminishing returns, (2) longer context history degrades performance, (3) chain-of-thought prompting hurts rather than helps, and (4) a vision-language architecture provides an advantage at 7B but hurts at 32B. Because the same model's performance ranges from 20% to 57% depending on interface parameters alone, these results characterize the agent-plus-interface system, not the model in isolation. Under this unified zero-shot ASCII protocol, current open-weight LLM agents remain well below approximate rodent reference values, particularly on tasks requiring spatial navigation and within-trial state tracking.

• The paper introduces CheeseBench, a novel benchmark evaluating LLMs on nine rodent behavioral tasks modeled after classical neuroscience protocols.
• It employs zero-shot protocols with fixed action spaces and ASCII renderings to probe procedural learning, emphasizing differences in spatial navigation and working memory.
• Empirical results show LLMs excel in simple stimulus-response tasks but fall short of rodent-level performance in spatial and memory-intensive challenges.

Evaluating LLMs on Rodent Behavioral Neuroscience Paradigms: Insights from CheeseBench

Introduction and Motivation

CheeseBench introduces a rigorously constructed benchmark for evaluating LLM agents on tasks derived directly from classical rodent behavioral neuroscience protocols. Unlike declarative benchmarks, CheeseBench targets procedural discovery, goal-learning through trial and error, spatial navigation, and instrumental conditioning—a suite of cognitive skills underrepresented in conventional AI evaluation. The benchmark consists of nine tasks (Morris Water Maze, Barnes Maze, T-maze, Radial Arm Maze, Star Maze, Operant Chamber, Shuttle Box, Conditioned Place Preference, Delayed Non-Match to Sample), grounded in peer-reviewed protocols and associated with reference animal performance baselines.

All environments are rendered as ASCII gridworlds and presented under a unified zero-shot protocol, requiring agents to infer goals and strategies exclusively from raw text observations and sparse reward signals, without task-specific instruction. This setup mimics the cognitive challenges faced by rodents and directly probes the capacity of LLMs for procedural, reward-driven learning.

Figure 1: ASCII renderings of all nine CheeseBench environments as seen by the model, including distinct spatial layouts and agent markers.

Benchmark Protocol and Design

CheeseBench standardizes agent-environment interaction: agents receive only the environment’s ASCII rendering, with a fixed action space (FORWARD, ROTATE_LEFT, ROTATE_RIGHT, STAY), and must respond with up to $k$ actions plus a free-text "learnings" segment serving as explicit working memory. Observation windows (history length), action batch sizes, prompt variants, and input modality (ASCII/text vs. vision) are parameterized and systematically ablated.

Three ASCII observation modes are available—top-down (full-grid), egocentric-cropped, and pseudo-3D first-person views (Figure 2), with the top-down grid used as default for all main results.

Figure 2: The three ASCII observation modes for the Star Maze: top-down, egocentric, and pseudo-3D, underscoring the agent’s perspective variation.

The primary evaluation focuses on six open-weight LLMs (3B–72B parameters, predominantly Qwen2.5-VL family), compared against random and Tabular Q-Learning baselines and graph search oracles. Task success is defined by the ability to reach goal states or perform required actions, mapped to binary outcomes directly comparable to standardized rodent learning curve asymptotes.

Empirical Results

Model Performance and Task Difficulty

Success rates expose marked gradients across both models and behavioral domains. The best-performing configuration (Qwen2.5-VL-7B) achieves a mean 52.6% success across tasks, compared to 32.1% for a random baseline and 78.9% for approximate rodent performance estimators.

Per-task performance reveals that LLM agents approach rodent-level on some tasks requiring simple stimulus-response associations (e.g., 94% on Operant Chamber, 95% on Shuttle Box), but perform significantly worse than rodents on spatial memory (e.g., <50% on Morris Water Maze, Barnes Maze) and are nearly incapable on working memory-dependent tasks (e.g., 5% on Radial Arm Maze).

Figure 3: Per-environment success rates for all agents; Radial Arm Maze remains essentially unsolved by all configurations, exposing critical working memory limitations.

Cognitive Profiling

Analysis across six cognitive domains exposes fundamentally distinct agent strengths. Tabular-QL solves reinforcement-driven conditioning but fails on spatial and working memory tasks; LLM agents display the reverse trend. Notably, even the largest LLMs fall far below rodents in spatial and memory-based challenges, even as they outperform classical RL agents in flexibility and some generalization scenarios.

Figure 4: Cognitive profile Radar plot – LLMs handle a broader range of tasks, with clear deficits in working memory/spatial tasks as opposed to Tabular-QL.

Scaling, Interface Sensitivity, and Ablations

Scaling LLM size from 3B to 7B delivers a ≥2x improvement in global success rate, but further scaling to 32B/72B saturates with negligible improvement (Figure 5). This scaling break contrasts typical NLP benchmarks and suggests limits stemming from interface and memory, not representational capacity.

Figure 5: Scaling behavior – performance plateaus sharply beyond 7B under the unified protocol, indicating non-capacity bottlenecks dominate.

Ablation studies (Figure 6) reveal that performance is highly non-monotonic with context history length (shorter is reliably better), chain-of-thought prompting consistently degrades performance, and vision-language encoders assist on text input only at the 7B scale (not at 32B). The default prompt (succinct, strategy-oriented) and batch action size $k=4$ yield optimal results.

Figure 6: Ablation results demonstrating the critical dependence of performance on interface history length, prompt, action batch size, and the interaction with vision-language encoders.

Theoretical and Practical Implications

CheeseBench demonstrates that procedural, reward-driven discovery—particularly where within-episode state tracking and spatial mapping are necessary—remains unsolved for LLM-based agents, even with large-scale, multimodal transformer architectures. The inability of LLMs to match rodent performance in tasks requiring the construction and adaptive use of internal state highlights representational and interface bottlenecks. In contrast, tasks dominated by direct stimulus-response mappings are tractable, consistent with emergent in-context reinforcement phenomena documented elsewhere.

These findings implicate interface design as a critical determinant of observed agent competence. Success rates vary as much as 37pp for a fixed model under different history and action prompting protocols, demonstrating that the evaluation must encompass the agent-plus-interface rather than attributing failures or successes to underlying model “cognition” alone.

The saturation with scale suggests that fundamental architectural modifications or external memory mechanisms, rather than model scaling or routine prompt engineering, will be required to close the gap with biological reference performance in procedural and spatial cognition.

Future Directions

The paper underscores the relevance of animal-based behavioral paradigms as grounding benchmarks for artificial cognition. CheeseBench provides a reproducible, reference-anchored platform for further research in memory augmentation, reinforcement-driven policy learning, recurrent architectures, and more realistic agent-environment interfaces. Further exploration of intermediate memory models (e.g., per-action scratchpads, spatial working memory graphs), explicit episodic memory augmentation, or hybridized RL-LLM systems may yield substantial performance gains. Broader adoption of comparative psychology paradigms could foster the development of more grounded, robust forms of artificial general intelligence.

Conclusion

Current open-weight LLM agents, running under a unified zero-shot ASCII protocol, fail to match rodent-level cognitive performance on standardized behavioral neuroscience tasks. The observed deficits are most acute in navigation and within-trial state tracking, with clear empirical signatures in scaling, history-length dependence, and interface sensitivity. CheeseBench, therefore, reorients the focus from isolated LLM capability to holistic agent-environment-system evaluation and provides a biologically anchored roadmap for advancing the cognitive competence of artificial agents, inviting future research in both engineering and the scientific study of intelligence.

---

Reference:

"CheeseBench: Evaluating LLMs on Rodent Behavioral Neuroscience Paradigms" (2604.10825)