---
title: 'PsychoLexQuery: Persian Therapy Evaluation'
url: https://www.emergentmind.com/topics/psycholexquery
type: topic
---

# PsychoLexQuery: Persian Therapy Evaluation

Searching arXiv for recent papers directly relevant to PsychoLexQuery and its surrounding framework.
PsychoLexQuery is a **single-turn evaluation dataset** for **Persian psychotherapy-style dialogue**, introduced within the **PsychoLexTherapy** framework to test whether a model can answer a **real user’s mental-health question** with **empathy, coherence, cultural fit, and therapy-oriented reasoning** [2510.03913]. It is constructed from publicly available Persian counseling interactions rather than synthetic prompts, and it functions as the single-turn counterpart to the multi-turn **PsychoLexDialogue** resource in the same framework. In this setting, PsychoLexQuery is not a general psychological QA benchmark; it is a targeted evaluation resource for culturally grounded therapeutic response generation in Persian.

## 1. Definition and institutional role

PsychoLexQuery serves as the **single-turn benchmark** in a three-stage system development process consisting of: **assessing SLMs psychological knowledge with PsychoLexEval; designing and implementing the reasoning-oriented PsychoLexTherapy framework; and constructing two evaluation datasets—PsychoLexQuery and PsychoLexDialogue—to benchmark against multiple baselines** [2510.03913]. Its immediate purpose is to evaluate whether a small language model can respond appropriately to a one-shot counseling-style message without relying on multi-turn memory.

Within that design, PsychoLexQuery occupies a specific methodological niche. It evaluates response generation to a single user message, whereas **PsychoLexDialogue** extends the same problem into longer therapeutic interactions. The distinction is operationally important because, for the **single-turn benchmark, memory is inactive**, while in multi-turn evaluation the long-term memory module is activated. This means PsychoLexQuery isolates the contribution of therapeutic routing, structured reasoning, and culturally aligned response formulation from the separate contribution of structured memory.

The dataset is therefore best understood as an evaluation layer for **psychotherapy simulation** rather than as a corpus for general mental-health classification or ordinary instruction following. Its emphasis is on whether a generated answer is therapeutically coherent in Persian, not merely fluent or emotionally warm.

## 2. Data acquisition and anonymization

PsychoLexQuery was built from **real Persian user questions** collected from **Persian-language psychology forums and counseling platforms**, specifically **EhyaCenter+**, **Moshaverfa**, and **Simiaroom** [2510.03913]. The construction procedure used **web crawling** and **manual extraction** of publicly available posts in which users describe psychological concerns and counselors or psychologists respond.

The paper states three reasons for this sourcing strategy: **ecological validity**, because the questions are real rather than synthetic; **cultural authenticity**, because the language, concerns, and social context reflect Persian/Iranian user experience; and **therapy relevance**, because the prompts come from genuine counseling situations. These properties distinguish PsychoLexQuery from evaluation sets assembled from examination material or automatically generated scenarios.

Because the source material is sensitive, the preprocessing pipeline applied extensive anonymization. Names were replaced with placeholders such as **“Person A”**; locations were generalized into coarse categories such as **“large city”**; timestamps were abstracted to month/year level; phone numbers, emails, and social media handles were removed; and specific job or project identifiers were generalized. The stated goal was to preserve **cultural realism** while protecting privacy. This design makes the dataset suitable for model evaluation in a mental-health setting where confidentiality is not incidental but structurally central.

## 3. Size, topics, and task characteristics

The paper states that PsychoLexQuery contains **approximately 4,000 user questions** [2510.03913]. The topics span common mental-health concerns in Persian contexts, including **relationship difficulties, anxiety, depression, family conflict, academic pressure, workplace stress, self-esteem issues, social anxiety, grief and loss, and behavioral issues in children/adolescents**. Figure and table descriptions indicate that **relationship difficulties** were the dominant category and that multiple emotional themes commonly co-occurred in a single question.

This topical profile matters for interpretation. PsychoLexQuery is not limited to one diagnostic domain, one counseling school, or one user demographic scenario. Instead, it samples the heterogeneous problem space of counseling-style help seeking as it appears in Persian online platforms. A plausible implication is that strong performance on the dataset requires robust handling of mixed affective and interpersonal content rather than narrow optimization for one symptom label.

The task formulation is also narrower than generic open-ended dialogue. A system must answer a real counseling-style question in a way that is judged on **empathy**, **coherence and structure**, **cultural fit**, **therapeutic alignment**, **content accuracy**, **adaptability**, **linguistic fluency**, **clarity**, and **human-likeness** on a **10-point Likert scale** in the automatic evaluation protocol. That specification makes PsychoLexQuery a response-quality benchmark with explicit therapeutic and cultural criteria, not simply a next-turn prediction dataset.

## 4. Integration with PsychoLexTherapy

PsychoLexQuery is inseparable from the architecture it was designed to test. **PsychoLexTherapy** is intended to simulate how a therapist reasons, rather than merely generating generic empathetic text, by first selecting a therapeutic orientation and then following a **structured reasoning path** aligned with that orientation [2510.03913]. The framework explicitly models **Cognitive Behavioral Therapy (CBT)**, **Reality Therapy (RT)**, and **Person-Centered Therapy (PCT)**.

The routing mechanism is the **Therapy Approach Selector**. It chooses a path according to message cues: **CBT** for negative automatic thoughts, distortions, or hopelessness; **PCT** for empathy, acceptance, and emotional reflection; and **RT** for needs, responsibility, choices, and action. After selection, the system executes an explicit multi-step path. The **CBT** path performs extraction of automatic thoughts, inference of emotional consequences, projection of behavioral tendencies, generation of balanced alternatives, derivation of adaptive behaviors, and synthesis into a final response. The **RT** path identifies needs and wants, analyzes current behaviors, evaluates consequences, plans alternative behaviors, and integrates them into a final response. The **PCT** path performs empathic reflection, exploratory questioning for self-awareness, and synthesis into a warm, non-directive response.

In the PsychoLexQuery setting, these structured paths are evaluated without long-term memory. That design choice is methodologically important because it tests whether therapeutic structure alone improves single-turn response quality. The broader framework uses **small language models** rather than large cloud-only systems. Candidate models were restricted to **under 10 billion parameters** for **technical feasibility** and **privacy**, with model families including **Gemma-3**, **Qwen-3**, **LLaMA-3.2**, and **Mistral**, and **Gemma-4B** chosen as the base model because it balanced **sufficient psychological competence**, **manageable compute requirements**, and **local deployment/privacy**. This suggests that PsychoLexQuery was designed for evaluation under privacy-preserving, on-device therapeutic simulation constraints rather than purely maximal-scale inference.

## 5. Evaluation protocol and reported findings

The single-turn PsychoLexQuery benchmark compares PsychoLexTherapy with four baselines: **Simple Prompt**, **Simple Prompt + Therapy Selector**, **Empathy Chain**, and **Empathic Agents** [2510.03913]. The **Simple Prompt** baseline uses a generic empathetic prompt with no explicit therapeutic reasoning. **Simple Prompt + Therapy Selector** adds routing to CBT/RT/PCT logic without a full structured reasoning chain. **Empathy Chain** uses a two-phase process consisting of a reasoning trace followed by a final empathetic answer, with **CBT**, **PCT**, and **RT** variants. **Empathic Agents** uses **multi-agent debate**, where multiple therapy-aligned agents generate candidate responses, critique one another, and a final decision agent synthesizes the output.

Automatic evaluation uses **GPT-5** as an **LLM-as-a-judge**. In the single-turn setting, outputs are rated on a **10-point Likert scale** across the nine criteria already listed. The study also includes a human preference evaluation in which **3 psychology graduate students**, blinded to system identity, were shown **5 anonymized outputs** and asked to rank them from **1 (best)** to **5 (worst)**.

The reported result is that **PsychoLexTherapy outperformed all baselines** on automatic LLM-as-a-judge evaluation and was also **ranked highest** in the human study [2510.03913]. The paper characterizes the baseline pattern as follows: **Simple Prompt** was fluent but weak therapeutically; **Simple + Selector** was slightly better but still shallow; **Empathy Chain** sometimes improved reasoning but could become mechanical or verbose; **Empathic Agents** showed strong performance due to multiple perspectives; and **PsychoLexTherapy** achieved the best overall balance of **empathy, structure, cultural fit, and therapeutic alignment**.

The authors attribute this outcome to the combination of **explicit therapeutic routing**, **structured reasoning paths**, **Persian cultural constraints**, and **internal quality-control logic**. In this interpretation, PsychoLexQuery operationalizes a distinction between generic empathy and therapy-oriented reasoning: success on the dataset requires not only emotionally resonant language but also mode selection and structured response organization.

## 6. Relation to the broader PsychoLex ecosystem and research significance

PsychoLexQuery belongs to a broader line of psychology-focused LLM resources but occupies a different layer of the stack. The earlier **PsychoLex** framework introduced **PsychoLexQA**, **PsychoLexEval**, and **PsychoLexLLaMA** as bilingual Persian-English resources for psychology-specific instruction tuning and multiple-choice evaluation; **PsychoLexEval** contains **3,430 rows** and is designed to test **psychological knowledge and reasoning** rather than psychotherapy-style response generation [2408.08848]. In contrast, PsychoLexQuery evaluates responses to **real Persian user questions** in a counseling setting. The difference is methodological: PsychoLexEval emphasizes domain knowledge and MCQA-style correctness, whereas PsychoLexQuery emphasizes therapeutic response quality under culturally grounded conditions.

This distinction aligns with a broader trend in psychological LLM research. **Psyche-R1** argues that reliable psychological systems should jointly integrate **empathy**, **psychological expertise**, and **reasoning**, rather than optimizing only one or two of these properties [2508.10848]. PsychoLexQuery can be read as an evaluation resource built around that same triad, but specialized for Persian psychotherapy-style dialogue and for **single-turn** assessment. This suggests that the dataset’s significance lies less in raw scale than in the specificity of what it measures: therapeutic appropriateness under real-world Persian mental-health prompting.

The limitations are also explicit. PsychoLexQuery is a **single-turn** dataset, so it does not directly measure continuity, personalization over time, or long-horizon therapeutic coherence. Those dimensions are addressed instead by **PsychoLexDialogue**, which is built by expanding PsychoLexQuery into **3,400 dialogues**, **16 thematic categories**, and an average of **10–14 turns**. More generally, the benchmark evaluates psychotherapy simulation, not clinical efficacy, and the surrounding work does not claim readiness for autonomous clinical care.

In encyclopedic terms, PsychoLexQuery is most accurately characterized as a culturally grounded Persian evaluation dataset for one-shot psychotherapeutic response generation. Its defining properties are the use of real user questions, strong anonymization, explicit linkage to therapy-specific reasoning paths, and evaluation protocols that privilege empathy, coherence, cultural fit, and therapeutic alignment over generic fluency alone [2510.03913].

Source: https://www.emergentmind.com/topics/psycholexquery