TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents (1901.08149v2)

Abstract: We introduce a new approach to generative data-driven dialogue systems (e.g. chatbots) called TransferTransfo which is a combination of a Transfer learning based training scheme and a high-capacity Transformer model. Fine-tuning is performed by using a multi-task objective which combines several unsupervised prediction tasks. The resulting fine-tuned model shows strong improvements over the current state-of-the-art end-to-end conversational models like memory augmented seq2seq and information-retrieval models. On the privately held PERSONA-CHAT dataset of the Conversational Intelligence Challenge 2, this approach obtains a new state-of-the-art, with respective perplexity, Hits@1 and F1 metrics of 16.28 (45 % absolute improvement), 80.7 (46 % absolute improvement) and 19.5 (20 % absolute improvement).

TransferTransfo: A Transfer Learning Approach for Neural Network-Based Conversational Agents

TransferTransfo presents a novel method for enhancing generative conversational agents by utilizing transfer learning with a high-capacity Transformer model. This paper outlines a significant advancement in the state-of-the-art for non-goal-oriented dialogue systems. The core innovation lies in the integration of a multi-task objective during fine-tuning, which amalgamates multiple unsupervised prediction tasks to achieve significant improvements over traditional methods, including memory-augmented seq2seq and information-retrieval models.

Methodology

The model employs a 12-layer Transformer encoder, heavily inspired by the Generative Pre-trained Transformer (GPT) architecture. The approach involves two primary steps: pre-training and fine-tuning. Pre-training is conducted on the BooksCorpus dataset, preserving the document-level corpus format to leverage long-range contexts effectively. This step utilizes the model weights provided by prior work on transfer learning, enabling the model to capture intricate language features that are vital for dialogue generation.

Fine-tuning is executed using the {\sc persona-chat} dataset and incorporates a novel input representation to handle two-speaker dialogue settings. The methodology includes two key elements:

1. Input Representation: Tokens are crafted by concatenating personality sentences with previous dialogue history. An additional set of dialog-state embeddings is appended to this token sequence, distinguishing between different conversational aspects like personality sentences or speakers.
2. Multi-task Learning: Fine-tuning involves optimizing a combined loss function, mixing next-utterance classification loss with a LLMing loss, following a strategy similar to BERT’s Next Sentence Prediction task.

Evaluation and Results

The proposed method was rigorously evaluated using the {\sc persona-chat} dataset, which includes challenging metrics such as perplexity (PPL), Hits@1, and F1 scores. TransferTransfo achieved unparalleled success, attaining a substantial 45% improvement in perplexity, 46% in Hits@1, and 20% in F1 on the test set. These metrics indicate a robust enhancement in model fluency, relevance, and ability to maintain coherent personality traits across dialogues.

Implications and Future Work

These findings demonstrate that transfer learning can significantly bolster the performance of generative dialogue models, a domain traditionally dominated by discriminative language tasks. The demonstrated effectiveness suggests further exploration into multi-task training configurations and different pre-training datasets might elucidate additional insights into optimizing generative dialogue systems.

Future research directions could include a deeper examination of personalized dialogue generation, where models not only maintain a consistent persona but adapt dynamically to more nuanced conversation flows. The method's applicability across diverse languages and cultural contexts also presents a rich vein for further exploration, potentially transforming how conversational agents are deployed in multilingual and multicultural settings.

In conclusion, TransferTransfo contributes a substantial leap forward in generative conversational agents, reinforcing the prominence of transfer learning paradigms within the field of Natural Language Processing.