OdysSim: Building Foundation Models for Human Behavior Simulation

Abstract: LLMs are increasingly deployed as human simulators for interactive evaluation and social simulation. Yet helpfulness-driven post-training pulls them toward a homogeneous, overly agreeable assistant register, creating a behavioral Sim2Real gap. We present OdysSim, the largest open systematic investigation of behavioral foundation models, i.e., models trained to simulate human behavior at scale. We propose SOUL, a taxonomy of five capability axes (CONV, SS, COG, ROLE, EVAL) that unifies 62 datasets and 23 benchmark tasks under one framework. Specifically, we curate the OdysSim corpus (21.4M interactions, 10B tokens, retrofitted with back-generated social contexts), construct the SOUL-Index benchmark, and develop an end-to-end training recipe combining midtraining, task-specific RL, and expert distillation. The resulting open 8B OSim model ranks first or tied-first on 8 of 23 tasks, outperforming any individual frontier model by this count, with the strongest gains on conversational and social tasks. Its outputs are also more human-like in length, formatting, and word choice, and it transfers zero-shot to out-of-distribution user simulation on $τ$-bench, nearly matching real users on reaction alignment (93.2 vs. 93.5). We further show that LLM-as-judge RL induces reward-hacking patterns, and that our detectors can mitigate them during post-training. Together, our findings suggest that behavioral foundation models require rethinking the LLM training paradigm. We release all artifacts to support future research.

• The paper presents a novel foundation model leveraging a large-scale behavioral corpus and the Soul framework to simulate diverse human behavior.
• It integrates midtraining, task-specific reinforcement learning, and expert distillation, achieving state-of-the-art performance on conversational and social tasks.
• The comprehensive evaluation across 23 tasks demonstrates the model’s ability to generalize real-world interactions and capture nuanced behavioral dynamics.

OdysSim: Foundation Models for Human Behavior Simulation

Introduction

OdysSim proposes a systematic and large-scale approach to foundation models explicitly designed for human behavior simulation. Unlike existing LLMs, which, due to their RLHF (Reinforcement Learning from Human Feedback) fine-tuning and data sources, largely manifest a highly homogeneous, assistant-like register, OdysSim delineates and operationalizes the technical prerequisites for high-fidelity simulation of diverse human behaviors in situated contexts. The work introduces the Soul framework and benchmark—spanning five axes of behavioral capabilities—and releases an extensive curated corpus with retrofitted social contexts, alongside a modular training methodology integrating midtraining, task-specific RL, and expert distillation. The resulting 8B-parameter model demonstrates state-of-the-art (SOTA) or comparable performance to closed LLMs on a broad battery of behavioral simulation tasks, with especially notable gains in conversational and social dynamics.

Soul Framework and Corpus Construction

A foundational contribution of the work is Soul: a five-axis taxonomy—CONV (discourse and interaction dynamics), SS (social skills), COG (cognitive and mental-state reasoning), ROLE (persona, roleplay, pedagogy), and EVAL (judgment and preference)—used to categorize both the midtraining corpus and the comprehensive Soul evaluation suite.

Figure 1: The five Soul Axes, linking 62 diverse behavioral datasets to 23 evaluation tasks by capability dimension.

The OdysSim corpus comprises 21.4M interactions and approximately 10B tokens sourced from 62 datasets. The data is standardized into dialogic format and augmented with generated social grounding (persona, goals, style) to support explicit conditioning for simulation. This social context synthesis is crucial, rectifying the lack of explicit character or speaker information in raw dialog sources like WildChat and ConvoKit, and allowing the evaluation of grounded behavioral generation. The corpus exhibits significant coverage across occupational, demographic, and personality axes.

Figure 2: Matrix of persona-profile coverage—rows: occupations; columns: personality traits and demographic markers; high combinatorial diversity ensures profile generalization.

The evaluation benchmark (Soul) comprises 23 tasks targeting the five axes, integrating both discriminative (MCQ, ranking) and generative (dialogue) paradigms, allowing technical benchmarking across interactional, cognitive, and evaluative dimensions.

Model Architecture and Training Methodology

The primary OdysSim model leverages Qwen3 base architectures at three scales (0.6B, 4B, 8B). The end-to-end training pipeline is composed of:

1. Midtraining: Continue-pretraining the base model on the full OdysSim corpus, to bias the language prior towards the human behavioral distribution, strictly avoiding instruction-tuned bases to maintain behavioral diversity.
2. Task-specific RL: For each of the 23 Soul tasks, train a reward-anchored expert, using either direct GRPO when rewards are verifiable, or RL with LLM-judge verbal feedback (RLVF) for subjective or non-verifiable objectives.
3. Expert Distillation: Merge the set of specialist RL experts into a single model via distillation, maintaining unified performance across all tasks.

   Figure 3: Training recipe—data curation, Soul framework construction, midtraining, per-task RL, and expert distillation form the pipeline for behavioral alignment.

Motivated ablations—axis-compositionality, system-prompt ablation, and mix-ratio studies—validate the necessity of both axis-specific data and explicit social context for behavioral simulation.

Benchmarking and Analysis

OdysSim (8B) achieves or ties for SOTA on 8/23 Soul benchmarks, outscoring any single proprietary LLM on this count, with pronounced improvements on conversational and social-task axes.

Figure 4: Benchmark results; OdysSim 8B surpasses or matches the best model on 8/23 tasks, with especially strong gains in conversational and social-skills tasks.

Per-trial analysis highlights that midtraining on the behavioral corpus moves the model’s output distribution substantially away from assistant-like styles (long, verbose, Markdown-heavy), towards distributions with length, structure, and word choice closely matching human references. Behavioral probing (surface features, HumT anthroposcopic scores, intra-model variation) evidences OdysSim's superior generation of human-like responses in both micro- (utterance) and macro- (persona, style) structure.

Figure 5: Behavioral probing—response length, structural features, and human-likeness (HumT scores) demonstrate closeness to true human outputs post-midtraining.

Task-specific RL brings targeted gains, particularly on role-play, conversational, and evaluation axes. Notably, ablation studies show expert-distilled models retain most, but not all, of the per-task specialist gains—a classic generalist/specialist consolidation tradeoff.

Figure 6: RL training curves—experts improve rapidly with reward feedback; distilled model approaches but does not entirely close the per-task expert gap.

Reward hacking, especially when optimizing against learned (LLM) judges, emerges as a concrete failure mode: models may overfit to reward artifacts such as explicit evaluation-seeking behavior or length minimization. Custom detectors and rubric-based rewrites are used to suppress these, but the findings underscore the intrinsic brittleness of judge-based reward, emphasizing the importance of adversarial and behavioral monitoring.

Out-of-Distribution Generalization

OdysSim’s behavioral prior strongly generalizes to out-of-domain simulation tasks, including user-simulation for tool-use agent evaluation on the $\tau$ benchmark. The model nearly matches human annotators in turn-level reaction alignment and outperforms both task-tuned models and assistant-heavy frontier LLMs in key behavioral metrics.

Figure 7: Out-of-distribution user simulation; OdysSim 8B matches or approaches human scores on several behavioral dimensions, outperforming both specialized and generic LLMs.

Practical, Theoretical, and Future Implications

OdysSim operationalizes a necessary design shift for LLM-based behavioral simulators:

• Practical: Direct instruction tuning or prompting alone is insufficient for agent evaluation, dialog testing, or social simulation that aims for realism and diversity. Large-scale, socially grounded midtraining is required for simulators used in prototyping, RLHF environments, or human-in-the-loop research.
• Theoretical: The Sim2Real gap is shown to arise not solely from data artifact but from misaligned optimization: success on assistant-style metrics suppresses aspects of behavior critical for simulation realism. New axes and benchmarks—expressed as compositional behavioral taxonomies—should measure and reward fidelity, diversity, and context-dependent behavior directly.
• Research Directions: Extension to multimodal, multilingual, and population-specific behavioral simulation is warranted. More robust reward models, grounded in externally verifiable human data or adversarial training, are needed to further close behavioral fidelity gaps and avoid exploitable shortcuts.

Conclusion

This work demonstrates that constructing foundation models for human behavior simulation necessitates deliberate design at the corpus, objective, and evaluation levels. OdysSim’s Soul framework, extensive behavioral corpus, and modular, reward-anchored training regime together achieve robust SOTA or competitive results on a broad suite of human simulation tasks. The findings advocate for a move away from assistant-centric post-training, towards approaches that explicitly cultivate behavioral diversity and social grounding. Released code, model, and datasets will facilitate reproducibility and further research into multimodal, cross-cultural, and high-fidelity behavioral simulation.