NL2Domain: Mapping Language to Domain Outputs

• NL2Domain tasks are computational problems that convert natural language inputs into specialized outputs like labels, entity annotations, or DSL representations.
• They employ techniques such as domain adaptation, semantic parsing, and weak supervision, achieving notable performance improvements measured by metrics like F1 scores.
• Applications range from information extraction and dialogue systems to technical text understanding and automated planning, ensuring domain-specific robustness.

Natural Language to Domain (NL2Domain) tasks comprise a family of computational problems in which natural language input (typically at the sentence or document level) is automatically mapped to a set of domain-specific outputs. These outputs may range from domain labels, domain-sensitive named entity annotations, and slot/value extractions, to the synthesis of executable scripts, plans, or structured representations within a formal domain-specific language (DSL). NL2Domain sits at the intersection of domain adaptation, semantic parsing, weak supervision, and zero-/few-shot learning. The area has emerged as a crucial subfield for ensuring domain transferability and specificity in practical NLP systems, with applications in information extraction, dialogue systems, technical text understanding, and automated knowledge base construction.

1. Core Problem Classes and Definitions

The NL2Domain paradigm encompasses distinct but structurally related instantiations, including:

• Domain identification: Classification of a text span into predefined or open-ended domain ontologies based on observable cues (e.g., genre, topic, technical field).
• Domain adaptation for IE/NER: Synthesis and transfer of extraction models (e.g., NER) from high-resource to low-/zero-resource domains, often relying on weak or automatically generated supervision.
• Semantic parsing to domain-specific formal languages: Translation of natural-language task statements to compositional DSLs (e.g., Bash, PDDL) aligned with domain grammars.
• Domain-sensitive task completion: End-to-end mapping from language to actionable domain-specific outputs (e.g., API calls or plans), often with out-of-domain robustness constraints.

Formally, most NL2Domain tasks involve learning a mapping

$f_{\theta}: \mathcal{X} \to \mathcal{Y}_d,$

where $\mathcal{X}$ are natural language inputs and $\mathcal{Y}_d$ are structured outputs conditioned on domain $d$ (which may be predicted, selected, or constructed during inference).

2. Approaches to Domain Labeling and Detection

Zero-shot domain labeling exploits large pre-trained LLMs to assign one or more domain/category labels to a text fragment, usually without explicit in-domain fine-tuning. One paradigm, demonstrated in "Ask2Transformers: Zero-Shot Domain labelling with Pre-trained LLMs" (Sainz et al., 2021), recasts domain assignment as an NLI problem: the candidate gloss and a label-injecting hypothesis are jointly scored for entailment. With robust prompt engineering, e.g., “The domain of the sentence is about [label],” state-of-the-art F1 up to 92.14% is achieved on WordNet synset labeling (BabelDomains/WordNet).

Weakly supervised domain detection, as in DetNet (Xu et al., 2019), utilizes hierarchical transformer encoders and MIL, propagating distant supervision from documents down to sentences and words to recover fine-grained, multilabel domain evidence. The framework produces domain scores at multiple levels (word/sentence/document), and achieves document-level F1 up to 76.5%, with transfer to out-of-genre datasets.

Evaluation datasets such as TGeGUM (“Can Humans Identify Domains?” (Barrett et al., 2024)) reveal that domain is an inherently fuzzy, gradient notion—human genre agreement peaks at $\kappa \approx 0.66$ (contextual prose), and fine-grained topic assignments remain challenging (F1 $<$ 20% for 100 topics). Pretrained transformer classifiers (e.g., DeBERTaV3) approach human performance on coarse labels but exhibit sharp performance drops with increased granularity.

3. Domain Adaptation and Transfer for Structured IE

Domain transfer in IE tasks, especially NER, frequently relies on automated supervision generation and cross-domain knowledge transfer. The pipeline in "Domain-Transferable Method for Named Entity Recognition Task" (NL2Domain Task) (Mikhailov et al., 2020) is emblematic:

• Construct a large domain-specific entity vocabulary by crawling Wikipedia categories.
• Generate "silver-standard" labels on an unlabelled target domain corpus by unifying (a) a general-domain NER model’s predictions and (b) exact-match morphology-based annotations.
• Discard all-O sentences and duplicates; merge pseudo-labeled data with source NER data.
• Domain-adapt the encoder with MLM pre-training on the target corpus.
• Fine-tune for NER on the merged pseudo/canonical data.

In the Russian History domain (no human-labeled target data), this yields micro-averaged F1 of 0.80–0.87. The domain-adapted RuBERT model, in conjunction with silver labels, exhibits significant improvements in MISC/PER labels relative to purely generic NER.

Multi-task learning and open-vocabulary RNN slot-filling models ("Domain Adaptation of Recurrent Neural Networks for Natural Language Understanding" (Jaech et al., 2016)) further reduce data requirements via parameter sharing and character-level generalization, particularly benefitting low-resource or OOV-exposed slots.

4. NL2Domain for Semantic Parsing and Executable Synthesis

NL2Domain methodologies generalize naturally to semantic parsing of NL into executable DSLs. The NL2Bash/NL2CMD pipeline typifies this, mapping English to Bash (Lin et al., 2018, Fu et al., 2023):

• Construct/expand domain-specific grammars (e.g., from man-pages), generate large paired corpora ($>$70k command–NL pairs) via code synthesis, execution-based validation, and back-translation.
• Train NMT-style models (e.g., Transformers with 6+6 layers, ensemble checkpoints, label smoothing) on both canonical and synthesized data.
• Evaluate with strict structural criteria (exact utility/flag match; execution correctness).

Top-1 exact-structure accuracy reaches 53.2% on the canonical NL2Bash set and 31.6% on synthesized examples (Fu et al., 2023). Error sources are dominated by utility misclassification ($\sim$67%), flag confusion (33%), and domain shift between real and synthetic distributions.

The general approach—syntactic code generation, validation by execution or static analysis, and active dataset construction—readily extrapolates to arbitrary DSLs given accessible grammar and execution environments.

5. Domain Identification in Dialogue and Planning Systems

NL2Domain also subsumes domain-sensitive dialogue and planning:

• Task-oriented Dialogue: ZeroToD (Mosharrof et al., 18 Feb 2025) conditions open-source LLMs on schema descriptions (with augmentation via systematic synonymization of slot/method names) to induce API-calling behaviour from raw dialogs, yielding best-in-class Complete API Accuracy on unseen (out-of-domain) tasks—FLAN-T5 with schema augmentation rises from 53.16% to 61.07%. The system is trained solely on raw dialog and schema, without turn-level annotation.
• Domain-aware Dialogue Generation: DOM-Seq2Seq (Choudhary et al., 2017) uses a domain-classifier (ensemble logistic regression or RNN over prior utterance domains and SVM TF-IDF predictions) to select among domain-specific decoders. Classification accuracy (on Reddit/Twitter) peaks at 77.57%, and domain-informed reranking consistently improves generation metrics.
• Planning from Text: NL2Plan (Gestrin et al., 2024) incrementally extracts PDDL domain/problem files from free-form input using LLM-driven chain-of-thought prompting, strict syntactic and semantic validation, and classical planning. Success rate surpasses direct LLM plan-generation (66.7% vs 13.3%), with failure detection regimes that preempt unsound outputs.

6. Benchmarking, Evaluation, and Analysis

NL2Domain methods utilize a variety of evaluation approaches:

• Supervised classification: Macro/micro-averaged precision, recall, F1 across classes (e.g., per-label and overall for NER and domain ID).
• Structural/execution correctness: Exact-structure, utility/flag, or template accuracy for code synthesis; command execution validity.
• Human and machine agreement: Inter-annotator agreement (Fleiss’ kappa), distributional similarity, multi-annotation regression/classification.
• Long-form outputs and factuality: In methodical tasks (DOLOMITES (Malaviya et al., 2024)), BLEU, ROUGE-L, BLEURT, section presence, and round-trip NLI entailment.

Reported results universally show a gap between cross-domain/zero-shot performance and supervised in-domain upper bounds, and the recurring importance of (a) domain schema access, (b) context-rich modeling, and (c) explicit handling of uncertainty and drift.

Method/Task	Domain Supervision	Best F1 / Accuracies	Key Attributes
Zero-shot NLI LM	None	F1 92% (A2T+desc.)	Prompt engineering critical
DetNet-MIL	Weak labels	Doc F1 76%	Multi-granular, cross-lingual
NER RuBERT-adapted	Silver (auto-annot.)	F1 80–87%	Domain-adapt. + merged pseudo/sup.
NL2Bash/NL2CMD	Programmatic synth	Acc 31–53%	DSL extension, validation, ensemble
ZeroToD (FLAN-T5)	No dialog annotation	61% (out-of-domain)	Schema aug., LLMs, API gen.
NL2Plan	LLM+chain-of-thought	67% (planning tasks)	Full DSL gen., feedback, validator

7. Limitations, Open Problems, and Future Directions

Despite strong progress, several challenges remain:

• Human-level ambiguity: Empirical disagreement on “domain” highlights limits of discrete labeling; effective pipelines should accommodate soft/multilabel and uncertainty-aware outputs (Barrett et al., 2024).
• Fine-grained domain distinctions and rare classes: Performance drops sharply with increasing label/slot granularity and in the low-resource regime; approaches such as hierarchical classification, data synthesis, and active learning are vital.
• Unsupervised/semi-supervised expansion: Automatic label/slot induction, alias expansion, and domain-aware data augmentation (e.g., LLM-conditioned paraphrasing, back-translation) remain promising but require robust error filtering (Mikhailov et al., 2020, Fu et al., 2023).
• Evaluation metrics for structure and domain factuality: Long-form, structured outputs (DOLOMITES, NL2Plan) lack strong reference-based or automatic correctness metrics; new NLI-, QA-, or retrieval-based measures are needed (Malaviya et al., 2024, Gestrin et al., 2024).
• Explainability and assistive workflows: Blended human-in-the-loop and explainable intermediate artifacts (NL2Plan) are essential for complex domains where errors can propagate or be ambiguous.

Ongoing directions include chain-of-thought modeling for procedural tasks, cross-lingual and cross-modal extension, and benchmark expansion to emerging expert and creative domains.

---

• "Domain-Transferable Method for Named Entity Recognition Task" (Mikhailov et al., 2020)
• "Ask2Transformers: Zero-Shot Domain labelling with Pre-trained LLMs" (Sainz et al., 2021)
• "NL2Bash: A Corpus and Semantic Parser for Natural Language Interface to the Linux Operating System" (Lin et al., 2018)
• "NL2CMD: An Updated Workflow for Natural Language to Bash Commands Translation" (Fu et al., 2023)
• "Domain Aware Neural Dialog System" (Choudhary et al., 2017)
• "Weakly Supervised Domain Detection" (Xu et al., 2019)
• "DOLOMITES: Domain-Specific Long-Form Methodical Tasks" (Malaviya et al., 2024)
• "Domain Adaptation of Recurrent Neural Networks for Natural Language Understanding" (Jaech et al., 2016)
• "Multilingual Pre-Trained Transformers and Convolutional NN Classification Models for Technical Domain Identification" (Dowlagar et al., 2021)
• "Can Humans Identify Domains?" (Barrett et al., 2024)
• "Evaluating and Enhancing Out-of-Domain Generalization of Task-Oriented Dialog Systems for Task Completion without Turn-level Dialog Annotations" (Mosharrof et al., 18 Feb 2025)
• "NL2Plan: Robust LLM-Driven Planning from Minimal Text Descriptions" (Gestrin et al., 2024)