AgentSME: A Unified Agent Framework

• AgentSME is a unified generative agent framework powered by LLMs that simulates learning through varied communication modes like Solo, Mono, and Echo.
• It operationalizes self-study, one-way tutoring, and peer collaboration using standardized models, prompts, and CMMLU sociology questions.
• Empirical findings reveal that Echo mode enhances accuracy and reasoning diversity, suggesting bidirectional peer exchange benefits learning performance.

AgentSME denotes, in its explicit published formulation, a unified generative agent framework powered by LLMs for smart education, designed to simulate how different communication patterns among agents affect learning performance and reasoning diversity. The framework treats each LLM as a “virtual student agent,” fixes the underlying models, prompts, datasets, and metrics, and varies only the communication mode among three directional forms—Solo, Mono, and Echo—corresponding to self-directed reasoning, unidirectional mentoring, and bidirectional peer co-learning. In that formulation, AgentSME is evaluated on CMMLU sociology questions and is used to study how agency autonomy and communicative reciprocity shape both answer accuracy and the diversity of chain-of-thought reasoning (Yang et al., 5 Aug 2025).

1. Problem formulation and conceptual scope

AgentSME is motivated by a specific gap in smart education research: most LLM-based educational systems either simulate a tutor answering a student’s questions or simulate virtual students for role-play and practice, but largely assume a single dialogue style and optimize for human-like interaction and content fidelity rather than for the effects of interaction structure on learning capability. The framework therefore asks a narrower and more technical question: if LLM agents are treated as learners, and their communication pattern is systematically varied, how does that alter their accuracy and the diversity of their reasoning (Yang et al., 5 Aug 2025)?

This framing places communication directionality and reciprocity at the center of the design space. The supplied educational rationale is that real pedagogical settings exhibit multiple communication forms—self-study, lecture, tutoring, and peer discussion—and that learners display diverse cognitive behaviors such as exploration, self-explanation, imitation, and peer collaboration. AgentSME abstracts this diversity into three directional modes and evaluates them under otherwise standardized conditions. A common misconception is to interpret the framework as a generic tutoring system; the published formulation is narrower. It is a controlled comparative framework for generative agents as learners, not a full classroom simulator or a longitudinal student model.

The paper also uses “unified” in a precise sense. The same underlying LLMs and question sets are used across all conditions, the agents are treated uniformly as virtual students, and only the communication mode changes. That methodological choice makes communication structure the principal explanatory variable rather than model identity, dataset variation, or prompt heterogeneity.

2. Agent model, architecture, and execution assumptions

In AgentSME, every LLM is abstracted as a student agent that receives a question, produces chain-of-thought reasoning denoted RC, and outputs a final answer Y in $\{A, B, C, D\}$. The common task input is the Society subset of CMMLU in single-choice format. Generation settings are standardized at temperature $= 0.7$ and top-p $= 0.95$, and context is reset after each question, so there is no cross-question memory; any apparent “learning” arises only within the interaction surrounding a single question (Yang et al., 5 Aug 2025).

Capacity tier	Models
High-capacity	GPT-4o; DeepSeek-Reasoner; Qwen-Plus
Base-capacity	GPT-3.5-Turbo; DeepSeek-Chat; Qwen-Turbo-Latest

All three instantiated models—AgentSolo, AgentMono, and AgentEcho—share the same underlying machinery: LLM endpoints, question set, and evaluation metrics. What differs is prompt construction and the sharing of intermediate reasoning among agents. The core implicit functions are BuildPrompt(a_i, q_k), GeneAnswer or GeneResponse, and UpdateAnswer(a_i, RC_peer). The last function appends peer reasoning to the original question context and instructs the agent to reconsider and answer again.

The framework deliberately excludes several design elements often associated with more elaborate agent systems. It does not implement explicit symbolic internal states, planning modules, or external tools. Those capacities are treated as implicit in the LLM’s chain-of-thought. This matters interpretively: AgentSME is not a tool-augmented, memory-rich, planner-executor architecture. It is a prompt-mediated comparative framework for interaction structure. A plausible implication is that its findings isolate communication effects more cleanly, but at the cost of simplifying many features of real learners and production educational agents.

3. Directional communication modes

The three communication modes differ in autonomy and reciprocity, and each is explicitly tied to a pedagogical analogue (Yang et al., 5 Aug 2025).

Mode	Operational definition	Pedagogical analogue
Solo	Each agent reasons in isolation	Self-study / self-explanation
Mono	A mentor provides reasoning once; the learner revises	Lecture or one-way tutoring
Echo	Two agents exchange reasoning and both revise	Peer discussion / collaborative learning

In Solo mode, each agent processes the question independently. If $a_i$ denotes an agent and $q$ a question, the framework builds a prompt $ASK_i = BuildPrompt(a_i, q)$ and generates

$(RC_i, Y_i) = GeneAnswer(a_i, ASK_i).$

No other agent’s reasoning or answer is visible, and context is cleared after the answer. This mode functions as the baseline for inherent accuracy and lexical diversity.

In Mono mode, two roles are defined: mentor and learner. The mentor independently produces reasoning,

$RC_m = GeneResponse(a_m, ASK_i),$

and the learner then receives the mentor’s full reasoning and updates its own response:

$(RC_i, Y_i) = UpdateAnswer(a_i, RC_m).$

The mentor does not revise. Mono therefore models one-way knowledge transfer without mutual adaptation. An important clarification is that Mono is not an interactive tutoring loop; it is a single-shot consultation mechanism.

In Echo mode, two agents reason independently, exchange reasoning, and both revise:

$$(RC_i, Y_i) = UpdateAnswer(a_i, RC_j), \qquad (RC_j, Y_j) = UpdateAnswer(a_j, RC_i).$$

The supplied notes state that the pseudo-code in the paper contains a minor inconsistency, but the intended semantics are clear: both agents first reason about the same question, then each updates after reading the other’s reasoning. Echo is therefore the only mode with bidirectional influence and mutual revision. The paper treats it as a model of peer-to-peer mutual assistance and as an exploration of the upper bound of dialogue-based collaboration for generative agents.

4. Evaluation design and analytical measures

The evaluation uses the Society subset of CMMLU, comprising 264 single-choice sociology questions spanning fundamental concepts, classical theories, and research methods. Before comparing communication modes, the study estimates question difficulty by running all six LLMs for five independent Solo-style rounds on all 264 questions, computing average error rates, and assigning difficulty levels according to fixed thresholds: HIGH if average error rate $= 0.7$0, MEDIUM if error rate is in $= 0.7$1, and LOW if error rate is in $= 0.7$2. The reported analyses then use 20 HIGH, 21 MEDIUM, and 44 LOW questions (Yang et al., 5 Aug 2025).

For each difficulty level and each of the six LLMs, the study conducts 10 independent repeats, $= 0.7$3. In Solo, one agent answers independently. In Mono, a learner receives mentor reasoning and re-answers. In Echo, two agents powered by different LLMs are randomly paired to exchange reasoning and re-answer. The text does not enumerate the exact pairing schedule, but states that pairing is random and repeated across rounds for robustness.

Accuracy is the primary evaluation metric. The paper also reports best run and standard deviation and uses statistical comparisons with reported $= 0.7$4-values, often $= 0.7$5. To measure diversity of reasoning content rather than correctness alone, the study defines three lexical diversity indices over the corpus of reasoning traces RC:

1. Inverse Simpson Index

$= 0.7$6

2. Honoré’s Statistic

$= 0.7$7

3. Information Entropy

$= 0.7$8

Here, $= 0.7$9 is the total number of tokens, $= 0.95$0 the number of unique word types, $= 0.95$1 the frequency of word type $= 0.95$2, $= 0.95$3 the number of hapax legomena, and $= 0.95$4 the empirical probability of word $= 0.95$5. The metrics are normalized by z-score within each mode, transformed by the standard normal CDF to $= 0.95$6, and visualized as radar charts. The important methodological point is that diversity is operationalized as lexical diversity of reasoning traces, not as diversity of final answers or pedagogical strategies.

5. Empirical findings

Across all difficulty levels and models, the dominant pattern is AgentEcho $= 0.95$7 AgentMono $= 0.95$8 AgentSolo, with Echo consistently achieving the highest accuracy and especially large gains on hard questions (Yang et al., 5 Aug 2025).

For HIGH-difficulty questions, the reported Solo mean accuracies for high-capacity models are GPT-4o: 0.035, Qwen-Plus: 0.120, and DeepSeek-Reasoner: 0.120, while Echo raises these to 0.367, 0.587, and 0.502 respectively. The reported gains are roughly +0.25 to +0.47 over Solo, all with $= 0.95$9. Among base-capacity models, Qwen-Turbo-Latest increases from 0.050 in Solo to 0.521 in Echo, and GPT-3.5-Turbo from 0.085 to 0.284. By contrast, AgentMono shows only marginal improvements over Solo on high difficulty, often $a_i$0, supporting the paper’s conclusion that single-shot unidirectional mentoring is insufficient for difficult reasoning errors.

For MEDIUM-difficulty questions, Solo accuracies range from 0.107 to 0.684, whereas Echo ranges from 0.475 to 0.849. Specific examples include Qwen-Plus improving from 0.395 to 0.849 and DeepSeek-Chat from 0.580 to 0.806. For LOW-difficulty questions, Solo ranges from 0.218 to 0.857 and Echo from 0.570 to 0.958. Examples include Qwen-Plus moving from 0.857 to 0.958 and DeepSeek-Chat from 0.766 to 0.937. On easier questions, Mono remains very similar to Solo, which the paper interprets as evidence that inherent LLM capability is already sufficient and additional communication offers limited marginal gain.

Echo’s gains are accompanied by greater variability. The paper reports higher standard deviation under Echo, reaching up to 0.187 on hard questions. That pattern matters because it rules out a simplistic interpretation of bidirectional exchange as purely stabilizing. Echo can strongly improve performance, but peer interaction can also mislead, yielding more run-to-run variance.

The diversity analysis shows a different but related pattern. In Solo mode, Qwen-Plus has the most balanced and high diversity scores, while DeepSeek-Reasoner is very low. In Mono, Qwen-Plus remains strong and balanced, DeepSeek-Reasoner shows a sharp improvement, and Qwen-Turbo-Latest regresses. In Echo, DeepSeek-Reasoner has the largest, most symmetrical radar shape and reaches the outermost bounds across the diversity metrics; this is the basis for the paper’s conclusion that DeepSeek exhibits the greatest diversity. GPT-3.5-Turbo remains compact, indicating limited responsiveness to feedback. The paper explicitly notes that diversity does not correlate linearly with accuracy: GPT-4o is highly accurate with relatively stable, moderate diversity, whereas DeepSeek-Reasoner in Echo mode combines high accuracy with very high diversity.

6. Pedagogical significance, limitations, and broader uses of the label

The educational interpretation of AgentSME maps the three modes onto recognizable instructional forms. Solo corresponds to self-study or self-explanation, Mono to lecture or one-way tutoring, and Echo to peer discussion or collaborative learning. The paper links Echo to ICAP-style interactive engagement and argues that smart education systems should not be limited to a simple tutor-to-student pipeline; instead, they should allow agents to see and react to one another’s reasoning. It also notes that weaker or lower-capacity models benefit particularly from Echo-style collaboration, suggesting that structured peer interaction can compensate for limited standalone capability (Yang et al., 5 Aug 2025).

The framework also has explicit limits. It uses only sociology questions from CMMLU and only single-choice QA, not open-ended writing, multi-step problem solving, or multimodal tasks. Agents have no persistent memory across questions, so all “learning” is per-question adaptation. Human cognitive diversity is approximated through LLM differences and communication modes, which the supplied notes identify as still simplistic. Future directions proposed in the same materials include more communication modes, multi-party discussion, multi-step Echo, teacher-student-peer hybrids, multimodal data, persistent memory, classroom-scale simulations, and human-in-the-loop studies.

A further point of clarification concerns nomenclature. In the supplied literature, “AgentSME” is not confined to the smart-education framework. In a software-engineering context, the term is used as a synthesized label for an AI-native methodology of “subject-matter expert” multi-agent teams grounded in "Agentsway," even though that paper does not itself use the term as its formal title; there the emphasis is on a human orchestrator and specialized Planning, Prompting, Coding, Testing, and Fine-Tuning agents (Bandara et al., 26 Oct 2025). In a Semantic Web and Web of Things context, the supplied walkthrough frames an “AgentSME”-style environment around Ricci’s Agent & Artifact meta-model, CArtAgO, JaCaMo, Jason, Moise, W3C WoT Thing Descriptions, and a Usage Ontology for semantically described artifact operations (Daoud, 2020). In SME sustainability assessment, an AI-driven ESG workflow built on n8n, expert-validated FL549 baselines, quartile-based tiering, and LLM-generated recommendations is described as something that can be thought of as an AgentSME prototype (Trinh et al., 5 Apr 2026). This suggests that the label has begun to function as a portable shorthand for domain-specialized multi-agent systems, while its clearest explicit instantiation remains the smart-education framework centered on Solo, Mono, and Echo communication modes.