---
title: 'PARSE-10K: Persian QA & 3D Scene Datasets'
url: https://www.emergentmind.com/topics/parse-10k
type: topic
---

# PARSE-10K: Persian QA & 3D Scene Datasets

Searching arXiv for the cited PARSE/PARSE-10K papers to ground the article.
I’m going to look up recent arXiv entries related to “PARSE-10K” and “PARSE” to disambiguate the term and ensure the article is grounded in the current literature.
PARSE-10K is not a single standardized designation in the arXiv literature. The term is used in two primary senses: as an informal name for **PARSE**, an open-domain Persian reasoning question answering benchmark containing \(10{,}800\) questions, and as the formal name **PARSE-10K**, a dataset of \(10{,}000\) 3D indoor scenes produced within a part-aware spatial modeling framework. In other PARSE-titled works, the authors explicitly state that no PARSE-10K dataset or benchmark is defined there, which makes domain-level disambiguation essential when the label appears without context [2602.01246] [2603.07704] [2605.04263] [2510.08623].

## 1. Terminological scope and disambiguation

In current usage, “PARSE-10K” denotes two unrelated artifacts: a Persian reasoning QA resource and a 3D scene dataset. The shared string derives from the overloaded acronym **PARSE**, not from a common methodology or benchmark family. This terminological overlap is amplified by the fact that some papers use “PARSE” for speculative generation, schema optimization, or other tasks while explicitly noting that no “PARSE-10K” exists in those works. This suggests that the label should be treated as domain-specific rather than canonical [2605.04263] [2510.08623].

| Usage | Domain | Core definition |
|---|---|---|
| PARSE / “PARSE-10K” | Persian QA | Open-domain Persian reasoning QA benchmark with \(10{,}800\) questions |
| PARSE-10K | 3D vision and graphics | Dataset of \(10{,}000\) 3D indoor scenes with part-level contact structure |

The two usages differ not only in modality, but also in their unit of annotation, supervision regime, and evaluation protocol. One is a text QA benchmark organized by question type, reasoning regime, subtype, and difficulty; the other is a procedurally generated scene corpus organized around object parts, contact graphs, and spatial constraints. Treating them as interchangeable produces category errors in citation, benchmarking, and dataset comparison.

## 2. PARSE as an open-domain Persian reasoning QA benchmark

In the QA literature, **PARSE** is introduced as the first open-domain Persian reasoning QA benchmark and contains \(10{,}800\) questions across Boolean, multiple-choice, and factoid formats. It is designed for open-domain, cross-topic evaluation of reasoning-capable systems in Persian, a language spoken by roughly 130 million people. The benchmark simultaneously covers boolean questions, multiple-choice questions, factoid questions, multi-hop reasoning, general reasoning beyond simple retrieval, multi-answer cases, unanswerable cases, and explicit difficulty labels [2602.01246].

Its internal organization is a controlled cross-product. There are **3 question types**—Boolean, multiple-choice, and factoid—combined with **2 approach dimensions**—multihop and reasoning—and **3 subtypes per question type**. This yields \(3 \times 2 \times 3 = 18\) configurations. For each configuration, exactly **600** questions are generated, giving
\[
N_{\text{total}} = 18 \times 600 = 10{,}800.
\]
The benchmark is also balanced by main type and by approach dimension. Boolean, multiple-choice, and factoid each contribute \(3{,}600\) questions, while multihop and reasoning each contribute \(5{,}400\).

The subtype system is semantically heterogeneous. Boolean questions are divided into **simple**, **negation**, and **comparative** subtypes. Multiple-choice questions are divided into **single-answer**, **multi-answer**, and **non-answerable** items, with the “None” option as the only correct answer in the non-answerable case. Factoid questions are divided into **simple**, **list-based**, and **non-answerable** items. This design makes the benchmark closer to a composite evaluation suite than to a single-format QA dataset.

Difficulty is balanced as well. Each configuration contains **200 easy**, **200 medium**, and **200 hard** questions, so the complete benchmark contains \(3{,}600\) easy, \(3{,}600\) medium, and \(3{,}600\) hard items. The paper reports that human difficulty evaluation shows annotators’ accuracy is highest for easy questions, then medium, then hard, which confirms the practical meaning of the labels.

## 3. Construction, validation, and empirical findings for the Persian benchmark

The Persian PARSE benchmark is built with a controlled **LLM-based generation pipeline** using **GPT-4o** as the generator. Rather than using one generic prompt, the authors define **configuration-specific prompting**, so a setting such as Boolean–multihop–negation–hard has its own template. The prompts encode QA type and subtype, whether multihop and/or reasoning is required, difficulty balance, Persian-only output, multiple-choice formatting constraints, cross-topic realism, and answer-count constraints for multi-answer and list-based items. Generation proceeds in batches of **30 questions per LLM run**—10 easy, 10 medium, 10 hard—until **600** items are obtained for each configuration [2602.01246].

After generation, outputs are exported as CSV, concatenated per configuration, and converted to a normalized JSON schema containing at least `id`, `question`, `answer`, `difficulty`, and, for multiple-choice questions, exactly four `options`. The assembly process also tracks question type, approach, and subtype. Automatic filtering then enforces structural validity, de-duplication, semantic consistency, and linguistic robustness. The filtering rules are explicit: multiple-choice items must have exactly 4 options, multiple-choice multi-answer questions must have **2–4** correct options, factoid list-based items must have **2–5** gold answers, and hard questions must derive hardness from reasoning or semantic subtlety rather than obscure vocabulary or convoluted grammar.

Human evaluation is conducted in two stages. For quality evaluation, **10% of the dataset**, i.e. **1,080 QA pairs**, is sampled and partitioned into four non-overlapping groups of **270** items each. The annotator pool consists of **12 native Persian speakers (8 men, 4 women)** with educational backgrounds from undergraduate to master and ages **22–57**; each group is annotated by **3** annotators on **Ambiguity**, **Readability**, and **Correctness** using a **1–5** scale. The overall averages are **4.404** for Ambiguity, **4.669** for Readability, and **4.389** for Correctness. For difficulty evaluation, **270 questions** are sampled, and annotators answer them without seeing difficulty labels; observed accuracies follow the easy \(>\) medium \(>\) hard ordering.

Benchmarking covers multilingual and Persian LLMs, including **Qwen 2.5 (7B, 72B)**, **LLaMA 3 (8B, 70B)**, **Mistral 24B**, **Gemma 2 (27B)**, and the Persian-specialized **Dorna** model. The reported prompting pattern is consistent across evaluations: **Persian prompts usually outperform English prompts**, **chain-of-thought is often the best strategy for Boolean and multiple-choice tasks**, and **few-shot prompting usually yields the best performance for factoid tasks**. Fine-tuning uses **8,640** training questions and **2,160** test questions balanced across all **18** configurations, and the paper reports that fine-tuned **Dorna** achieves the best overall performance across configuration groups.

## 4. PARSE-10K as a part-aware 3D indoor scene dataset

In 3D vision and graphics, **PARSE-10K** is the dataset instantiation of **PARSE: Part-Aware Relational Spatial Modeling**. It consists of **10,000 unique 3D indoor scenes** constructed from real-image layout priors and a curated part-annotated shape database, with each scene containing dense contact structures and a part-level contact graph. The dataset is generated by a framework with three core components: the **Part-centric Assembly Graph (PAG)**, the **Part-Aware Spatial Configuration Solver**, and the dataset itself as the result of systematically instantiating PAGs with the solver [2603.07704].

The central representational object is the **PAG**, whose nodes are divided into **object nodes** and **part nodes**. Object nodes specify semantic entities such as chairs or tables, while part nodes specify subcomponents with labeled surfaces such as top, bottom, front, back, left, and right, defined with respect to each asset’s canonical pose. PAG edges are divided into **object-level spatial edges**, which encode coarse relations such as left of or behind, and **part-level geometric edges**, which encode preposition-labeled relations such as on, in, against, and aligned with.

The shape database supporting PARSE-10K is assembled by consolidating multiple public datasets with part annotations and part-segmented generative assets, producing a retrieval database covering **132 object categories**. The resulting dataset spans **17 room types** and is densely populated, with an average of **49.9 objects per scene**. The paper also states that the asset library contains over **17,372** part-segmented and semantically-labeled assets across those categories.

A key distinction from prior indoor scene datasets is annotation granularity. PARSE-10K provides **consistent part-level annotations** and **explicit physical relations**, including a dense part-level contact graph defined after physics refinement. This positions the dataset not merely as a scene collection, but as a structurally grounded corpus for learning object-level layout reasoning and part-level relation understanding.

## 5. Spatial solver, contact structures, and downstream use in 3D reasoning and generation

Scene synthesis in PARSE-10K is driven by the **Part-Aware Spatial Configuration Solver**. Given a PAG, the solver processes object nodes in a **topological sort**, following the sequential assembly order implied by support relations. Each object has a unique supporter, so the solving process begins within a **2D candidate region** on the supporter’s surface from which previously occupied areas have been excluded. Object-level relations such as “left of” further restrict this valid region [2603.07704].

The solver then instantiates a specific 3D asset with per-part segmentation and semantic labels, resolves part-level geometric edges, and formulates geometric constraints. If a relation specifies parts and surfaces directly, those are used. If only a preposition is given, the solver infers relevant parts and surfaces; for an **on** relation, this means identifying the asset’s lowest part and bottom surface while searching the target object for a suitable upward-facing support plane. After constraint accumulation, the solver samples a final pose from the feasible subspace, validates it for 3D collisions and physical-semantic plausibility, and refines the entire scene through a brief dynamic simulation in **Sapien**. From the stable configuration, the dataset derives a part-level contact graph by identifying all part pairs in close proximity, e.g. **\(\leq 1\) mm**.

PARSE-10K supports three VLM-oriented tasks: **Visual Relation MCQ**, **Part-level Contact MCQ**, and **Scene Graph Generation (SGG)**. In Visual Relation MCQ, the model answers multiple-choice questions about object-level spatial relations. In Part-level Contact MCQ, the model reasons over statements of the form “Part M of Object A contacts Part N of Object B.” In SGG, the model predicts object detections with labels and bounding boxes, together with relation triplets, and relations are counted as correct only if both subject and object are correctly matched detections and the predicted relation label matches ground truth.

Fine-tuning **Qwen3-VL** on PARSE-10K substantially improves performance. The paper reports **Visual Relation MCQ** accuracy increasing from **86.2%** to **97.4%**, **Part-level Contact MCQ** accuracy increasing from **60.4%** to **86.2%**, and **Scene Graph Generation** improving from **33.2/37.9 F1** to **76.6/78.2 F1** depending on whether grounding via bounding-box matching is enforced. PARSE-10K is also used to train a diffusion-based 3D scene generator. In a user study with **20 participants**, the **PAG-conditioned** model is preferred on **Complexity (47.5%)**, **Realism (38.8%)**, and **Contact Fidelity (45.0%)**, outperforming both **InstructScene** and an unconditioned PARSE-10K-trained variant.

## 6. Related uses, explicit non-uses, and scholarly significance

The ambiguity of “PARSE-10K” is sharpened by several nearby works that use the acronym **PARSE** while explicitly denying the existence of any PARSE-10K resource in their setting. The speculative generation framework **PArallel pRefix Speculative Engine** states that “PARSE-10K” is not a concept, dataset, or benchmark defined in that paper, and the schema-optimization framework **Parameter Automated Refinement and Schema Extraction** states that it does not mention “PARSE-10K” anywhere. These clarifications are important because acronym reuse alone does not imply dataset continuity or conceptual relatedness [2605.04263] [2510.08623].

From a bibliographic perspective, the name therefore has two legitimate research meanings and several illegitimate projections. In Persian NLP, “PARSE-10K” is a convenient shorthand for a **\(10{,}800\)-question** reasoning benchmark whose official name is **PARSE**. In 3D vision and graphics, **PARSE-10K** is the formal name of a **\(10{,}000\)-scene** dataset centered on part-aware spatial modeling. The two resources are linked only by acronymic coincidence.

The scholarly significance of the term lies in what each resource contributes within its own field. The Persian benchmark fills a low-resource evaluation gap by combining open-domain coverage, reasoning emphasis, multi-format QA, multi-answer and unanswerable cases, and difficulty control in a single benchmark. The 3D dataset advances geometry-grounded scene understanding by linking object-level layouts to part-level contacts, solver-based physical plausibility, and structured supervision for VLMs and generation models. In both cases, the “10K” scale marker signals breadth, but the underlying research agendas remain entirely separate.

Source: https://www.emergentmind.com/topics/parse-10k