---
title: 'ConvLab-3: Modular Toolkit for TOD Systems'
url: https://www.emergentmind.com/topics/convlab-3
type: topic
---

# ConvLab-3: Modular Toolkit for TOD Systems

ConvLab-3 is a flexible, modular toolkit designed for the rapid prototyping, training, and evaluation of task-oriented dialogue (TOD) systems. Central to ConvLab-3 is a unified data format, enabling seamless integration of diverse datasets and models. The framework provides robust reinforcement learning (RL) facilities, streamlined experimentation infrastructure, and a suite of user simulators, supporting both seasoned researchers and those new to dialogue system research [2211.17148].

## 1. Unified Data Format

Every dataset \(D\) within ConvLab-3 is organized into three core components: an ontology \(\mathcal{O}\), a set of dialogues \(\mathcal{T}\), and a database interface \(\mathrm{DB}\). The ontology specifies domains \(\mathcal{D} = \{d_1, \dots, d_{|\mathcal{D}|}\}\), with each domain possessing a slot set \(S_d\). Each slot \(s \in S_d\) is labeled categorical or non-categorical by a Boolean flag \(c_{d,s}\); categorical slots have a closed value set \(V_{d,s}\), otherwise \(V_{d,s}\) remains open.

A fixed set of intents \(\mathcal{I}\) and dialogue-act schemas 
\[
\mathcal{A}_\text{schema} \subseteq \mathcal{I} \times \mathcal{D} \times S_d \times (V_{d,s} \cup \{\emptyset\})
\]
define allowable semantic actions. An initial “empty” dialogue state is defined for each domain by \(\mathrm{state}_\mathrm{tmpl}(d) = \{(s, \bot) ~|~ s \in S_d\}\).

Each dialogue \(\tau \in \mathcal{T}\) is a sequence of \(T\) alternating user and system turns:
\[
\tau = \left\{(u_1, A^u_1), (r_1, A^r_1), \ldots, (u_T, A^u_T), (r_T, A^r_T)\right\}
\]
where \(u_t\) and \(r_t\) carry the utterance string, dialogue-acts \(A^u_t, A^r_t \subseteq \mathcal{A}_\text{schema}\), and a belief state \(b_t \in \prod_{d\in\mathcal{D}} (S_d \to V_{d,s}\cup\{\bot\})\), updated by \(b_t = \mathrm{Update}(b_{t-1}, A^u_t)\).

Database querying is performed via a standardized API
\[
\texttt{class BaseDatabase}~\{~\texttt{query(domain, state, k)}~\}
\]
allowing datasets with divergent schemas such as MultiWOZ and Schema-Guided Dialogue (SGD) to map into a unified format. This enables immediate cross-dataset model applicability without rewriting data-loading code.

## 2. System Architecture and Core Components

ConvLab-3 enforces a strictly modular pipeline at each dialogue turn. The sequence proceeds as follows:

1. User utterance
2. [NLU]: Natural Language Understanding maps utterance to discrete acts \(A^u\)
3. [DST]: Dialogue State Tracker updates belief state \(b_t = f_\text{DST}(b_{t-1}, A^u_t)\)
4. Database query for entities conditioned on \(b_t\)
5. [Vectoriser]: Encodes \((b_t, \mathrm{DB.res})\) for neural policy consumption
6. [Policy]: Maps vector inputs or belief state to system acts \(A^r_t = \pi(b_t)\)
7. [NLG]: Natural Language Generation produces the system’s textual utterance

Pseudocode for the dialogue loop:
```python
function RUN_DIALOGUE(agent, user_sim, max_turns):
    b = initialize_belief()
    for t in 1..max_turns:
        u_text = user_sim.respond(prev_sys_act, b)
        A_u = agent.NLU(u_text)
        b = agent.DST.update(b, A_u)
        db_res = agent.DB.query_all(b)
        vec_in = agent.Vectoriser(b, db_res, A_u)
        A_r = agent.Policy.act(vec_in)
        sys_text = agent.NLG.generate(A_r)
        if agent.Evaluator.is_success(b, A_u, A_r, db_res): break
    return agent.Evaluator.report()
```
This strict separation ensures that modules — including NLU, DST, policy, and NLG — can be replaced or combined arbitrarily.

## 3. Reinforcement Learning Infrastructure

ConvLab-3 formalizes dialogue management as a partially observable Markov decision process (POMDP),
\[
\mathcal{M} = \langle S, A, T, R, \gamma, O, \Omega \rangle
\]
with \(S\) as true dialogue states and user goals, \(A\) as system action sets, \(T\) as the user simulator-level transition function, \(R\) as the reward (success and per-turn penalties), and \(O\), \(\Omega\) as the observation space and model.

Reward functions commonly use:
\[
r_t = 
\begin{cases}
+20 & \text{if success at turn } t \\
-1 & \text{per turn penalty} \\
0 & \text{otherwise}
\end{cases}
\]
The toolkit directly supports Q-learning, policy-gradient, actor-critic (e.g., PPO), and off-policy/catastrophic forgetting mitigation methods (V-trace, CLEAR, DDPT). The RL training loop comprises data collection through user simulator interaction, policy updates (REINFORCE, PPO, etc.), and periodic evaluation on simulators or held-out human data.

## 4. User Simulators and Evaluation Metrics

ConvLab-3 incorporates several user simulators:
- ABUS: rule-based agenda
- TUS: Transformer-based act-only
- GenTUS: generative acts + NL
- EmoUS: GenTUS with emotion emission
- LLM-based: plug-in GPT/LLaMA via prompt

Swapping simulators is accomplished through configuration. Unified evaluation metrics include:
- Success rate: \(\text{SR} = \frac{1}{N} \sum_{i=1}^N \mathbb{1}\{\text{task}_i \text{ succeeded}\}\)
- Average turns: \(\bar{T} = \frac{1}{N} \sum_i T_i\)
- Average return: \(\bar{G} = \frac{1}{N} \sum_i G_i\)
- Action efficiency: \(\bar{a} = \frac{1}{N} \sum_{i,t} |A^r_{i,t}|\)
- Intent distribution: \(P(\mathrm{intent} = \alpha)\)

These are computed, logged, and reported by the unified evaluator, supporting systematic benchmarking.

## 5. Transfer Learning and Cross-Generalization

ConvLab-3 enables trivial cross-dataset transfer learning due to the unified format. Results from [2211.17148] highlight two key studies:

### 5.1 Supervised Pre-Train + Fine-Tune
Pre-training DST and NLG models on SGD + Taskmaster (~60k dialogs) then fine-tuning on MultiWOZ 2.1 (full, 1%, or 10%) yields significant gains even in low-resource regimes. For instance, SetSUMBT’s joint goal accuracy (JGA) on MultiWOZ 2.1 with 1% labeled data increases from 22.7 (no pre-training) to 43.8 (with pre-training). Similar trends are seen across larger data regimes.

### 5.2 RL-based Transfer
Applying the DDPT + V-trace algorithm under four regimes (no pre-training, SGD-only PT, 1% MWOZ only, SGD→1% MWOZ), transfer learning (SGD→1% MWOZ) achieves faster and higher convergence in strict success rate compared to from-scratch RL.

### 5.3 Cross-Simulator Policies

Strict success rates indicate strong simulator overfitting: a PPO-MLP trained on ABUS achieved 0.93 when tested on ABUS, but only 0.56 on GenTUS; policies often generalize poorly across simulators, an effect easily measured within ConvLab-3’s unified infrastructure.

| Train Simulator | Test: ABUS | Test: TUS | Test: GenTUS |
|:---------------:|:----------:|:---------:|:------------:|
| ABUS            | 0.93       | 0.71      | 0.56         |
| TUS             | 0.87       | 0.79      | 0.59         |
| GenTUS          | 0.89       | 0.86      | 0.63         |

*A plausible implication is that domain randomization or multitask approaches may be needed for robust generalization across user simulators in TOD settings.*

## 6. Usability, APIs, and Extensibility

Data loading, module configuration, and experiment execution are streamlined via Python APIs and YAML/JSON config files:

```python
from convlab.util import load_dataset, load_ontology, load_database

# load all splits
ds = load_dataset("multiwoz21")
onto = load_ontology("multiwoz21")
db = load_database("multiwoz21")

# few-shot
ds10 = load_dataset("multiwoz21", split2ratio={"train":0.1,"val":0.1})
```
Module composition is controlled via a JSON config, specifying `"class_path"` entries for components (e.g., NLU, DST, Policy, NLG, user policy), and invoked with:

```
$ convlab3 train --config config.json
$ convlab3 evaluate --config config.json
```

Adding new datasets requires conversion scripts that map raw JSON/XML to the unified ontology/dialogue schema; new models are integrated by subclassing relevant module interfaces and providing unified-format methods. This structure ensures that rapid prototyping, module swapping, and reproducibility are accessible by design, while not restricting researchers’ control over experiment granularity.

## 7. Significance and Impact

ConvLab-3’s unified data format, strictly modular system architecture, fully integrated RL training, diverse user simulator support, and transparent evaluation position it as a comprehensive platform for advancing task-oriented dialogue research. It facilitates comparative studies, generalization analyses, and transfer learning experiments with minimal friction, supporting both exploration of cutting-edge policies and educational use in new research contexts [2211.17148].

Source: https://www.emergentmind.com/topics/convlab-3