PARSE-10K: Persian QA & 3D Scene Datasets
- The paper on the Persian QA benchmark introduces 10,800 questions across Boolean, multiple-choice, and factoid types using a controlled LLM-based pipeline with balanced difficulty settings.
- PARSE-10K in 3D vision provides 10,000 indoor scenes with detailed part-level annotations and contact graphs, enhancing spatial reasoning and scene graph generation.
- Researchers must disambiguate 'PARSE-10K' as it denotes distinct resources in Persian NLP and 3D scene modeling, affecting citation, benchmarking, and application approaches.
Searching arXiv for the cited PARSE/PARSE-10K papers to ground the article. I’m going to look up recent arXiv entries related to “PARSE-10K” and “PARSE” to disambiguate the term and ensure the article is grounded in the current literature. PARSE-10K is not a single standardized designation in the arXiv literature. The term is used in two primary senses: as an informal name for PARSE, an open-domain Persian reasoning question answering benchmark containing questions, and as the formal name PARSE-10K, a dataset of 3D indoor scenes produced within a part-aware spatial modeling framework. In other PARSE-titled works, the authors explicitly state that no PARSE-10K dataset or benchmark is defined there, which makes domain-level disambiguation essential when the label appears without context (Mozafari et al., 1 Feb 2026, Bai et al., 8 Mar 2026, Yao et al., 5 May 2026, Shrimal et al., 8 Oct 2025).
1. Terminological scope and disambiguation
In current usage, “PARSE-10K” denotes two unrelated artifacts: a Persian reasoning QA resource and a 3D scene dataset. The shared string derives from the overloaded acronym PARSE, not from a common methodology or benchmark family. This terminological overlap is amplified by the fact that some papers use “PARSE” for speculative generation, schema optimization, or other tasks while explicitly noting that no “PARSE-10K” exists in those works. This suggests that the label should be treated as domain-specific rather than canonical (Yao et al., 5 May 2026, Shrimal et al., 8 Oct 2025).
| Usage | Domain | Core definition |
|---|---|---|
| PARSE / “PARSE-10K” | Persian QA | Open-domain Persian reasoning QA benchmark with questions |
| PARSE-10K | 3D vision and graphics | Dataset of 3D indoor scenes with part-level contact structure |
The two usages differ not only in modality, but also in their unit of annotation, supervision regime, and evaluation protocol. One is a text QA benchmark organized by question type, reasoning regime, subtype, and difficulty; the other is a procedurally generated scene corpus organized around object parts, contact graphs, and spatial constraints. Treating them as interchangeable produces category errors in citation, benchmarking, and dataset comparison.
2. PARSE as an open-domain Persian reasoning QA benchmark
In the QA literature, PARSE is introduced as the first open-domain Persian reasoning QA benchmark and contains questions across Boolean, multiple-choice, and factoid formats. It is designed for open-domain, cross-topic evaluation of reasoning-capable systems in Persian, a language spoken by roughly 130 million people. The benchmark simultaneously covers boolean questions, multiple-choice questions, factoid questions, multi-hop reasoning, general reasoning beyond simple retrieval, multi-answer cases, unanswerable cases, and explicit difficulty labels (Mozafari et al., 1 Feb 2026).
Its internal organization is a controlled cross-product. There are 3 question types—Boolean, multiple-choice, and factoid—combined with 2 approach dimensions—multihop and reasoning—and 3 subtypes per question type. This yields configurations. For each configuration, exactly 600 questions are generated, giving
The benchmark is also balanced by main type and by approach dimension. Boolean, multiple-choice, and factoid each contribute questions, while multihop and reasoning each contribute .
The subtype system is semantically heterogeneous. Boolean questions are divided into simple, negation, and comparative subtypes. Multiple-choice questions are divided into single-answer, multi-answer, and non-answerable items, with the “None” option as the only correct answer in the non-answerable case. Factoid questions are divided into simple, list-based, and non-answerable items. This design makes the benchmark closer to a composite evaluation suite than to a single-format QA dataset.
Difficulty is balanced as well. Each configuration contains 200 easy, 200 medium, and 200 hard questions, so the complete benchmark contains easy, 0 medium, and 1 hard items. The paper reports that human difficulty evaluation shows annotators’ accuracy is highest for easy questions, then medium, then hard, which confirms the practical meaning of the labels.
3. Construction, validation, and empirical findings for the Persian benchmark
The Persian PARSE benchmark is built with a controlled LLM-based generation pipeline using GPT-4o as the generator. Rather than using one generic prompt, the authors define configuration-specific prompting, so a setting such as Boolean–multihop–negation–hard has its own template. The prompts encode QA type and subtype, whether multihop and/or reasoning is required, difficulty balance, Persian-only output, multiple-choice formatting constraints, cross-topic realism, and answer-count constraints for multi-answer and list-based items. Generation proceeds in batches of 30 questions per LLM run—10 easy, 10 medium, 10 hard—until 600 items are obtained for each configuration (Mozafari et al., 1 Feb 2026).
After generation, outputs are exported as CSV, concatenated per configuration, and converted to a normalized JSON schema containing at least id, question, answer, difficulty, and, for multiple-choice questions, exactly four options. The assembly process also tracks question type, approach, and subtype. Automatic filtering then enforces structural validity, de-duplication, semantic consistency, and linguistic robustness. The filtering rules are explicit: multiple-choice items must have exactly 4 options, multiple-choice multi-answer questions must have 2–4 correct options, factoid list-based items must have 2–5 gold answers, and hard questions must derive hardness from reasoning or semantic subtlety rather than obscure vocabulary or convoluted grammar.
Human evaluation is conducted in two stages. For quality evaluation, 10% of the dataset, i.e. 1,080 QA pairs, is sampled and partitioned into four non-overlapping groups of 270 items each. The annotator pool consists of 12 native Persian speakers (8 men, 4 women) with educational backgrounds from undergraduate to master and ages 22–57; each group is annotated by 3 annotators on Ambiguity, Readability, and Correctness using a 1–5 scale. The overall averages are 4.404 for Ambiguity, 4.669 for Readability, and 4.389 for Correctness. For difficulty evaluation, 270 questions are sampled, and annotators answer them without seeing difficulty labels; observed accuracies follow the easy 2 medium 3 hard ordering.
Benchmarking covers multilingual and Persian LLMs, including Qwen 2.5 (7B, 72B), LLaMA 3 (8B, 70B), Mistral 24B, Gemma 2 (27B), and the Persian-specialized Dorna model. The reported prompting pattern is consistent across evaluations: Persian prompts usually outperform English prompts, chain-of-thought is often the best strategy for Boolean and multiple-choice tasks, and few-shot prompting usually yields the best performance for factoid tasks. Fine-tuning uses 8,640 training questions and 2,160 test questions balanced across all 18 configurations, and the paper reports that fine-tuned Dorna achieves the best overall performance across configuration groups.
4. PARSE-10K as a part-aware 3D indoor scene dataset
In 3D vision and graphics, PARSE-10K is the dataset instantiation of PARSE: Part-Aware Relational Spatial Modeling. It consists of 10,000 unique 3D indoor scenes constructed from real-image layout priors and a curated part-annotated shape database, with each scene containing dense contact structures and a part-level contact graph. The dataset is generated by a framework with three core components: the Part-centric Assembly Graph (PAG), the Part-Aware Spatial Configuration Solver, and the dataset itself as the result of systematically instantiating PAGs with the solver (Bai et al., 8 Mar 2026).
The central representational object is the PAG, whose nodes are divided into object nodes and part nodes. Object nodes specify semantic entities such as chairs or tables, while part nodes specify subcomponents with labeled surfaces such as top, bottom, front, back, left, and right, defined with respect to each asset’s canonical pose. PAG edges are divided into object-level spatial edges, which encode coarse relations such as left of or behind, and part-level geometric edges, which encode preposition-labeled relations such as on, in, against, and aligned with.
The shape database supporting PARSE-10K is assembled by consolidating multiple public datasets with part annotations and part-segmented generative assets, producing a retrieval database covering 132 object categories. The resulting dataset spans 17 room types and is densely populated, with an average of 49.9 objects per scene. The paper also states that the asset library contains over 17,372 part-segmented and semantically-labeled assets across those categories.
A key distinction from prior indoor scene datasets is annotation granularity. PARSE-10K provides consistent part-level annotations and explicit physical relations, including a dense part-level contact graph defined after physics refinement. This positions the dataset not merely as a scene collection, but as a structurally grounded corpus for learning object-level layout reasoning and part-level relation understanding.
5. Spatial solver, contact structures, and downstream use in 3D reasoning and generation
Scene synthesis in PARSE-10K is driven by the Part-Aware Spatial Configuration Solver. Given a PAG, the solver processes object nodes in a topological sort, following the sequential assembly order implied by support relations. Each object has a unique supporter, so the solving process begins within a 2D candidate region on the supporter’s surface from which previously occupied areas have been excluded. Object-level relations such as “left of” further restrict this valid region (Bai et al., 8 Mar 2026).
The solver then instantiates a specific 3D asset with per-part segmentation and semantic labels, resolves part-level geometric edges, and formulates geometric constraints. If a relation specifies parts and surfaces directly, those are used. If only a preposition is given, the solver infers relevant parts and surfaces; for an on relation, this means identifying the asset’s lowest part and bottom surface while searching the target object for a suitable upward-facing support plane. After constraint accumulation, the solver samples a final pose from the feasible subspace, validates it for 3D collisions and physical-semantic plausibility, and refines the entire scene through a brief dynamic simulation in Sapien. From the stable configuration, the dataset derives a part-level contact graph by identifying all part pairs in close proximity, e.g. 4 mm.
PARSE-10K supports three VLM-oriented tasks: Visual Relation MCQ, Part-level Contact MCQ, and Scene Graph Generation (SGG). In Visual Relation MCQ, the model answers multiple-choice questions about object-level spatial relations. In Part-level Contact MCQ, the model reasons over statements of the form “Part M of Object A contacts Part N of Object B.” In SGG, the model predicts object detections with labels and bounding boxes, together with relation triplets, and relations are counted as correct only if both subject and object are correctly matched detections and the predicted relation label matches ground truth.
Fine-tuning Qwen3-VL on PARSE-10K substantially improves performance. The paper reports Visual Relation MCQ accuracy increasing from 86.2% to 97.4%, Part-level Contact MCQ accuracy increasing from 60.4% to 86.2%, and Scene Graph Generation improving from 33.2/37.9 F1 to 76.6/78.2 F1 depending on whether grounding via bounding-box matching is enforced. PARSE-10K is also used to train a diffusion-based 3D scene generator. In a user study with 20 participants, the PAG-conditioned model is preferred on Complexity (47.5%), Realism (38.8%), and Contact Fidelity (45.0%), outperforming both InstructScene and an unconditioned PARSE-10K-trained variant.
6. Related uses, explicit non-uses, and scholarly significance
The ambiguity of “PARSE-10K” is sharpened by several nearby works that use the acronym PARSE while explicitly denying the existence of any PARSE-10K resource in their setting. The speculative generation framework PArallel pRefix Speculative Engine states that “PARSE-10K” is not a concept, dataset, or benchmark defined in that paper, and the schema-optimization framework Parameter Automated Refinement and Schema Extraction states that it does not mention “PARSE-10K” anywhere. These clarifications are important because acronym reuse alone does not imply dataset continuity or conceptual relatedness (Yao et al., 5 May 2026, Shrimal et al., 8 Oct 2025).
From a bibliographic perspective, the name therefore has two legitimate research meanings and several illegitimate projections. In Persian NLP, “PARSE-10K” is a convenient shorthand for a 5-question reasoning benchmark whose official name is PARSE. In 3D vision and graphics, PARSE-10K is the formal name of a 6-scene dataset centered on part-aware spatial modeling. The two resources are linked only by acronymic coincidence.
The scholarly significance of the term lies in what each resource contributes within its own field. The Persian benchmark fills a low-resource evaluation gap by combining open-domain coverage, reasoning emphasis, multi-format QA, multi-answer and unanswerable cases, and difficulty control in a single benchmark. The 3D dataset advances geometry-grounded scene understanding by linking object-level layouts to part-level contacts, solver-based physical plausibility, and structured supervision for VLMs and generation models. In both cases, the “10K” scale marker signals breadth, but the underlying research agendas remain entirely separate.