ConvLab-3: Modular Toolkit for TOD Systems

• ConvLab-3 is a modular toolkit designed for rapid prototyping, training, and evaluation of task-oriented dialogue systems using a unified data format.
• Its architecture splits key components like NLU, DST, policy, and NLG, enabling seamless integration and cross-dataset transfer with minimal code changes.
• The platform supports robust reinforcement learning with diverse user simulators and clear evaluation metrics, facilitating effective research and practical insights.

ConvLab-3 is a flexible, modular toolkit designed for the rapid prototyping, training, and evaluation of task-oriented dialogue (TOD) systems. Central to ConvLab-3 is a unified data format, enabling seamless integration of diverse datasets and models. The framework provides robust reinforcement learning (RL) facilities, streamlined experimentation infrastructure, and a suite of user simulators, supporting both seasoned researchers and those new to dialogue system research (Zhu et al., 2022).

1. Unified Data Format

Every dataset $D$ within ConvLab-3 is organized into three core components: an ontology $\mathcal{O}$, a set of dialogues $\mathcal{T}$, and a database interface $\mathrm{DB}$. The ontology specifies domains $\mathcal{D} = \{d_1, \dots, d_{|\mathcal{D}|}\}$, with each domain possessing a slot set $S_d$. Each slot $s \in S_d$ is labeled categorical or non-categorical by a Boolean flag $c_{d,s}$; categorical slots have a closed value set $V_{d,s}$, otherwise $V_{d,s}$ remains open.

A fixed set of intents $\mathcal{O}$0 and dialogue-act schemas

$\mathcal{O}$1

define allowable semantic actions. An initial “empty” dialogue state is defined for each domain by $\mathcal{O}$2.

Each dialogue $\mathcal{O}$3 is a sequence of $\mathcal{O}$4 alternating user and system turns: $\mathcal{O}$5 where $\mathcal{O}$6 and $\mathcal{O}$7 carry the utterance string, dialogue-acts $\mathcal{O}$8, and a belief state $\mathcal{O}$9, updated by $\mathcal{T}$0.

Database querying is performed via a standardized API

$\mathcal{T}$1

allowing datasets with divergent schemas such as MultiWOZ and Schema-Guided Dialogue (SGD) to map into a unified format. This enables immediate cross-dataset model applicability without rewriting data-loading code.

2. System Architecture and Core Components

ConvLab-3 enforces a strictly modular pipeline at each dialogue turn. The sequence proceeds as follows:

1. User utterance
2. [NLU]: Natural Language Understanding maps utterance to discrete acts $\mathcal{T}$2
3. [DST]: Dialogue State Tracker updates belief state $\mathcal{T}$3
4. Database query for entities conditioned on $\mathcal{T}$4
5. [Vectoriser]: Encodes $\mathcal{T}$5 for neural policy consumption
6. [Policy]: Maps vector inputs or belief state to system acts $\mathcal{T}$6
7. [NLG]: Natural Language Generation produces the system’s textual utterance

Pseudocode for the dialogue loop: $\mathcal{D} = \{d_1, \dots, d_{|\mathcal{D}|}\}$0 This strict separation ensures that modules — including NLU, DST, policy, and NLG — can be replaced or combined arbitrarily.

3. Reinforcement Learning Infrastructure

ConvLab-3 formalizes dialogue management as a partially observable Markov decision process (POMDP),

$\mathcal{T}$7

with $\mathcal{T}$8 as true dialogue states and user goals, $\mathcal{T}$9 as system action sets, $\mathrm{DB}$0 as the user simulator-level transition function, $\mathrm{DB}$1 as the reward (success and per-turn penalties), and $\mathrm{DB}$2, $\mathrm{DB}$3 as the observation space and model.

Reward functions commonly use: $\mathrm{DB}$4 The toolkit directly supports Q-learning, policy-gradient, actor-critic (e.g., PPO), and off-policy/catastrophic forgetting mitigation methods (V-trace, CLEAR, DDPT). The RL training loop comprises data collection through user simulator interaction, policy updates (REINFORCE, PPO, etc.), and periodic evaluation on simulators or held-out human data.

4. User Simulators and Evaluation Metrics

ConvLab-3 incorporates several user simulators:

• ABUS: rule-based agenda
• TUS: Transformer-based act-only
• GenTUS: generative acts + NL
• EmoUS: GenTUS with emotion emission
• LLM-based: plug-in GPT/LLaMA via prompt

Swapping simulators is accomplished through configuration. Unified evaluation metrics include:

• Success rate: $\mathrm{DB}$5
• Average turns: $\mathrm{DB}$6
• Average return: $\mathrm{DB}$7
• Action efficiency: $\mathrm{DB}$8
• Intent distribution: $\mathrm{DB}$9

These are computed, logged, and reported by the unified evaluator, supporting systematic benchmarking.

5. Transfer Learning and Cross-Generalization

ConvLab-3 enables trivial cross-dataset transfer learning due to the unified format. Results from (Zhu et al., 2022) highlight two key studies:

5.1 Supervised Pre-Train + Fine-Tune

Pre-training DST and NLG models on SGD + Taskmaster (~60k dialogs) then fine-tuning on MultiWOZ 2.1 (full, 1%, or 10%) yields significant gains even in low-resource regimes. For instance, SetSUMBT’s joint goal accuracy (JGA) on MultiWOZ 2.1 with 1% labeled data increases from 22.7 (no pre-training) to 43.8 (with pre-training). Similar trends are seen across larger data regimes.

5.2 RL-based Transfer

Applying the DDPT + V-trace algorithm under four regimes (no pre-training, SGD-only PT, 1% MWOZ only, SGD→1% MWOZ), transfer learning (SGD→1% MWOZ) achieves faster and higher convergence in strict success rate compared to from-scratch RL.

5.3 Cross-Simulator Policies

Strict success rates indicate strong simulator overfitting: a PPO-MLP trained on ABUS achieved 0.93 when tested on ABUS, but only 0.56 on GenTUS; policies often generalize poorly across simulators, an effect easily measured within ConvLab-3’s unified infrastructure.

Train Simulator	Test: ABUS	Test: TUS	Test: GenTUS
ABUS	0.93	0.71	0.56
TUS	0.87	0.79	0.59
GenTUS	0.89	0.86	0.63

A plausible implication is that domain randomization or multitask approaches may be needed for robust generalization across user simulators in TOD settings.

6. Usability, APIs, and Extensibility

Data loading, module configuration, and experiment execution are streamlined via Python APIs and YAML/JSON config files:

$\mathcal{D} = \{d_1, \dots, d_{|\mathcal{D}|}\}$1 Module composition is controlled via a JSON config, specifying "class_path" entries for components (e.g., NLU, DST, Policy, NLG, user policy), and invoked with:

$\mathcal{D} = \{d_1, \dots, d_{|\mathcal{D}|}\}$2

Adding new datasets requires conversion scripts that map raw JSON/XML to the unified ontology/dialogue schema; new models are integrated by subclassing relevant module interfaces and providing unified-format methods. This structure ensures that rapid prototyping, module swapping, and reproducibility are accessible by design, while not restricting researchers’ control over experiment granularity.

7. Significance and Impact

ConvLab-3’s unified data format, strictly modular system architecture, fully integrated RL training, diverse user simulator support, and transparent evaluation position it as a comprehensive platform for advancing task-oriented dialogue research. It facilitates comparative studies, generalization analyses, and transfer learning experiments with minimal friction, supporting both exploration of cutting-edge policies and educational use in new research contexts (Zhu et al., 2022).