Topic–Intent–Form Taxonomy

• Topic–Intent–Form taxonomy is a three-level structure that categorizes customer utterances into high-level topics, granular intents, and synthetic forms.
• It employs hierarchical topic modeling and LLM agents to expand static intent catalogs, addressing challenges like cold start and high term variability.
• The framework uses synthetic data generation and rigorous coherence metrics to ensure category distinctiveness and improve downstream intent recognition.

Searching arXiv for the specified paper to ground the article in the published record. Intent discovery and recognition in customer-service AI concerns the construction of a taxonomy that maps short, variable user utterances onto actionable semantic categories, and the subsequent training of recognizers that can assign new utterances to those categories. In "From Intent Discovery to Recognition with Topic Modeling and Synthetic Data" (Rodrigues et al., 16 May 2025), this problem is treated as a joint topic-modeling, taxonomy-expansion, and synthetic-data-generation task for domains marked by short utterances, infrequent word co-occurrences, high term variability, and the cold start problem. The resulting Topic–Intent–Form taxonomy combines hierarchical topic expansion, discovery of previously uncovered intents, and synthetic utterance generation into a three-level structure intended to support online discovery and recognition of new customer intents in dynamic domains (Rodrigues et al., 16 May 2025).

1. Problem setting and motivating constraints

The paper situates intent recognition in settings where recommender or customer-service systems must support new products or services without sufficient real user data, a condition identified as the cold start problem (Rodrigues et al., 16 May 2025). In this setting, customer utterances are short, exhibit infrequent word co-occurrences, and display high term variability, which complicates both the specification of distinct user needs and the preparation of synthetic queries.

The initial taxonomy consists of 36 human-curated, high-level banking topics, with examples such as "Open Account" and "Make Payment" (Rodrigues et al., 16 May 2025). The system’s purpose is not merely classification over a fixed label space, but expansion of that label space into more granular intents and the generation of utterance forms that can populate newly added classes. This suggests a shift from static intent catalogs toward continuously extensible taxonomies in which discovery and recognition are treated as coupled operations rather than separate stages.

A central premise is that traditional methods are stressed by short-query regimes, whereas LLM agents can contribute at multiple points: proposing subtopics, merging duplicates, identifying gaps from unlabeled data, and generating synthetic utterances (Rodrigues et al., 16 May 2025). The paper therefore frames LLMs not only as generators but as components in a structured workflow with explicit roles and validation criteria.

2. Hierarchical topic-modeling backbone

The first stage is described through the lens of a classical hierarchical topic model, even though the operational system relies on LLM agents for practical expansion (Rodrigues et al., 16 May 2025). The corpus is written as $\{w_{d,1},\dots,w_{d,N_d}\}_{d=1}^D$, with a two-level topic tree composed of level-1 topics $t\in\{1\dots T\}$ and, for each topic $t$, subtopics $u\in\{1\dots U_t\}$.

The simplified nested-LDA generative sketch is given as follows:

• Draw global topic proportions:

$$\boldsymbol\alpha\sim\mathrm{Dirichlet}(\gamma_1,\dots,\gamma_T).$$

• For each top-level topic $t$, draw subtopic weights:

$$\boldsymbol\beta^{(t)}\sim\mathrm{Dirichlet}(\delta_1,\dots,\delta_{U_t}).$$

• For each subtopic $u$ under $t$, draw word distribution:

$\phi_{t,u}\sim\mathrm{Dirichlet}(\eta).$

• For each document $t\in\{1\dots T\}$0:
  • Draw top-topic assignment:

  $t\in\{1\dots T\}$1 - Draw subtopic assignment:

  $t\in\{1\dots T\}$2 - For each word $t\in\{1\dots T\}$3 in document $t\in\{1\dots T\}$4, draw:

  $t\in\{1\dots T\}$5

Inference is described as proceeding by collapsed Gibbs sampling or variational Bayes, alternating over the conditional assignments:

$t\in\{1\dots T\}$6

and

$t\in\{1\dots T\}$7

Within the implemented workflow, the LLM-agent system is said to mimic these steps by generating candidate subtopics for each human-curated topic, filtering or merging semantically duplicate subtopics, and discovering entirely new subtopics from unlabeled data (Rodrigues et al., 16 May 2025). This creates a bridge between probabilistic topic-modeling intuition and agentic LLM orchestration: the former supplies the conceptual hierarchy, while the latter performs practical taxonomy operations on sparse, short-query data.

3. Evaluation criteria for topic quality

The topic-expansion process is evaluated using both traditional and LLM-based metrics (Rodrigues et al., 16 May 2025). For a topic’s top-$t\in\{1\dots T\}$8 words $t\in\{1\dots T\}$9, the paper uses Normalized Pointwise Mutual Information:

$t$0

It also uses $t$1 coherence via cosine-of-NPMI-vectors:

$t$2

$t$3

In addition, the paper reports LLM-based word- and document-level intruder detection and coherence ratings on a 1–3 scale, noting that these have shown higher correlation with human judgments (Rodrigues et al., 16 May 2025). The use of both distributional and LLM-mediated metrics is significant because topic quality here is not limited to lexical co-occurrence structure; it also concerns semantic distinctiveness and interpretability in a taxonomy that will later support synthetic data generation and intent recognition.

The before-and-after example given for the topic "open account" illustrates the operational meaning of improved specificity. The high-level label is expanded into granular intents such as savingsAccountOpen ("Open a savings account"), checkingAccountUpgrade ("Upgrade existing checking account"), and openBusinessAccount ("Open a new business checking account") (Rodrigues et al., 16 May 2025). This suggests that topic quality is being assessed not only by coherence but by the degree to which top-level categories can be refined into deployment-relevant intent classes.

4. Two-stage intent expansion workflow

The taxonomy expansion pipeline proceeds in two stages: Hierarchical Topic Expansion (HTE) followed by Topic Gap Bridging (TGB) (Rodrigues et al., 16 May 2025). Starting from the 36 human-curated topics, HTE first produces 222 intents, and TGB then discovers 56 new intents, yielding a total of 278 granular intents.

In HTE, the Intent Generator receives proxy-labeled utterances for each of the 36 topics and prompts an LLM to propose subtopics, together with descriptions, example queries, and a relevance score (Rodrigues et al., 16 May 2025). The Intent Merger then iteratively prompts an LLM to merge semantically identical subtopics under strict "rejection-sampling" criteria intended to ensure distinctiveness. The process terminates after 1,000 "no-merge" failures, and only 17 merges survived manual review (Rodrigues et al., 16 May 2025). The output of this stage is 222 intents under the 36 topics.

TGB is designed to discover intent classes not captured by the first pass. The Intent Proposer scans 200 unlabeled queries for out-of-domain examples and proposes new intents only when no existing subtopic applies (Rodrigues et al., 16 May 2025). The Intent Judge validates each proposed intent for distinctiveness and alignment, the Intent Refiner renames or refines approved intents to match taxonomy style, and the Examples Adder supplements intents lacking at least two examples via targeted selection from unlabeled data (Rodrigues et al., 16 May 2025). This second stage discovers 56 new intents, bringing the taxonomy to 278.

The sequence HTE$t$4TGB matters conceptually. HTE deepens existing topics, whereas TGB captures residual semantic mass not explained by the curated hierarchy. A plausible implication is that the workflow separates refinement of known semantic regions from explicit search for uncovered regions, which reduces the risk that novel intents are forced into the nearest existing class.

5. Synthetic query generation and the "Form" level

After intent discovery, the system generates synthetic user query data to augment real utterances and reduce dependency on human annotation in low-resource settings (Rodrigues et al., 16 May 2025). The paper states that once 144 intents are in place, synthetic utterances are generated per intent to fill the cold-start gap. These utterances constitute the "Form" level of the taxonomy.

Two prompting strategies are compared. In out-of-class prompting ("without in-class"), the prompt provides 10 few-shot examples drawn randomly from other intents (Rodrigues et al., 16 May 2025). In in-class few-shot prompting, the prompt adds 10 intent-specific examples (Rodrigues et al., 16 May 2025). Prompts enforce concise "search-bar style," chain-of-thought planning, structured JSON output, and batch-level conditioning for diversity (Rodrigues et al., 16 May 2025).

The study also compares human-curated intent descriptions and keywords against LLM-generated versions generated at $t$5 (Rodrigues et al., 16 May 2025). The reported finding is that synthetic intent descriptions yield equivalent synthetic-utterance quality to human descriptions, enabling fully automated pipelines. Within the paper’s framework, this means that human effort can be reduced not only in annotation but also in the authoring of the intent metadata used as generation context.

The final output of this stage is 100 utterances per intent (Rodrigues et al., 16 May 2025). These utterances function as synthetic surface forms attached to intent nodes, thereby expanding class coverage and variability. This suggests that the taxonomy is not only semantic but operational: each intent is linked to a set of realizations that can be used directly in recognizer training.

6. Topic–Intent–Form structure and empirical outcomes

The integrated taxonomy is a three-level tree composed of Topics, Intents/Subtopics, and Forms (Utterance Templates) (Rodrigues et al., 16 May 2025). The paper summarizes this structure as follows: Level 1: Topics (36 $t$6 278 via HTE+TGB), Level 2: Intents/Subtopics (for example, topic=openAccount $t$7 subtopics={openCheckingAccount, openSavingsAccount, \dots}), and Level 3: Forms generated synthetically, with 100 per intent (Rodrigues et al., 16 May 2025). Classifiers can then be trained to map form$t$8intent$t$9topic in a single or multi-stage architecture.

The empirical results report gains in topic modeling, intrinsic quality of synthetic data, and downstream intent recognition (Rodrigues et al., 16 May 2025).

Evaluation area	Setting	Reported result
Topic modeling	Topics	36 $u\in\{1\dots U_t\}$0 222 (HTE) $u\in\{1\dots U_t\}$1 278 (TGB)
Topic modeling	NPMI	$u\in\{1\dots U_t\}$2
Topic modeling	$u\in\{1\dots U_t\}$3	$u\in\{1\dots U_t\}$4
Topic modeling	Intruder (Doc-level)	$u\in\{1\dots U_t\}$5
Topic modeling	Coherence Rating (Doc-level)	$u\in\{1\dots U_t\}$6
Synthetic data	Distinct-N	$u\in\{1\dots U_t\}$7 without in-class vs. $u\in\{1\dots U_t\}$8 with in-class
Synthetic data	Compression Ratio	$u\in\{1\dots U_t\}$9
Synthetic data	Query Mean Specificity	$$\boldsymbol\alpha\sim\mathrm{Dirichlet}(\gamma_1,\dots,\gamma_T).$$0
Synthetic data	Discrimination Acc.	$$\boldsymbol\alpha\sim\mathrm{Dirichlet}(\gamma_1,\dots,\gamma_T).$$1 in all conditions
Intent recognition	Baseline (100% real)	$$\boldsymbol\alpha\sim\mathrm{Dirichlet}(\gamma_1,\dots,\gamma_T).$$2
Intent recognition	25% synthetic replacement (Approach 1)	$$\boldsymbol\alpha\sim\mathrm{Dirichlet}(\gamma_1,\dots,\gamma_T).$$3
Intent recognition	25% full-label replacement (Approach 2)	$$\boldsymbol\alpha\sim\mathrm{Dirichlet}(\gamma_1,\dots,\gamma_T).$$4–$$\boldsymbol\alpha\sim\mathrm{Dirichlet}(\gamma_1,\dots,\gamma_T).$$5
Intent recognition	Student’s $$\boldsymbol\alpha\sim\mathrm{Dirichlet}(\gamma_1,\dots,\gamma_T).$$6-test	$$\boldsymbol\alpha\sim\mathrm{Dirichlet}(\gamma_1,\dots,\gamma_T).$$7 between baseline and synthetic-only classes

These results are interpreted in the paper as showing that taxonomy expansion improves topic coherence and relevance, that in-class few-shot prompting yields more diverse and complex synthetic queries that better match real data, and that synthetic data augmentation can fully replace real data for some classes with only a modest drop in recognition performance, thereby addressing cold start (Rodrigues et al., 16 May 2025).

A common misconception would be to treat the synthetic pipeline as merely a paraphrase generator. The reported intrinsic metrics and downstream evaluations indicate a more specific role: the generated forms are intended to increase dataset variability and class coverage while preserving enough class fidelity that recognizers can be trained with no loss under partial synthetic replacement and only a modest decline under full-label replacement for some classes (Rodrigues et al., 16 May 2025). In that sense, the Topic–Intent–Form taxonomy is both an ontology of customer needs and a data-construction framework for intent recognition in dynamic, low-resource domains.