Learning to Write with Cooperative Discriminators (1805.06087v1)

Abstract: Recurrent Neural Networks (RNNs) are powerful autoregressive sequence models, but when used to generate natural language their output tends to be overly generic, repetitive, and self-contradictory. We postulate that the objective function optimized by RNN LLMs, which amounts to the overall perplexity of a text, is not expressive enough to capture the notion of communicative goals described by linguistic principles such as Grice's Maxims. We propose learning a mixture of multiple discriminative models that can be used to complement the RNN generator and guide the decoding process. Human evaluation demonstrates that text generated by our system is preferred over that of baselines by a large margin and significantly enhances the overall coherence, style, and information content of the generated text.

• The paper introduces a novel framework integrating cooperative discriminators with RNNs to enhance long-form text generation.
• It employs discriminators that focus on repetition, entailment, relevance, and lexical style to refine outputs using a Product of Experts approach.
• Empirical evaluations demonstrate significant improvements in coherence, clarity, and style over traditional RNN and GAN-based models.

Learning to Write with Cooperative Discriminators: An Expert Analysis

The paper "Learning to Write with Cooperative Discriminators" proposes a novel framework to improve the quality of long-form text generation using Recurrent Neural Networks (RNNs). The authors, a team from the University of Washington and the Allen Institute for Artificial Intelligence, address persistent issues in machine-generated text such as repetition, lack of coherence, and generic outcomes by leveraging a committee of cooperative discriminators. These discriminators are tasked with guiding the RNN-based generator to produce more globally coherent and contextually suitable continuations.

Methodology and Framework

The core contribution lies in the integration of multiple discriminative models that collectively refine the output of a base RNN LLM. Inspired by Grice's maxims of communication, these discriminators focus on principles like quantity, quality, relation, and manner to critique and enhance the text generation. The composite decoding objective combines the RNN generator with these discriminators, which receive individual weightings that the model learns and adjusts dynamically.

Key components of the framework include:

1. Repetition Model: This discriminator minimizes redundancy by learning to distinguish between RNN-generated and gold-standard continuations. It achieves this by assessing pairwise cosine similarity of word embeddings over a sliding window.
2. Entailment Model: This model reduces contradictions and redundant statements by using a natural language inference approach, trained on large-scale entailment datasets such as SNLI and MultiNLI. It scores against Grice's maxims and penalizes contradictions or straightforward entailments.
3. Relevance Model: This model weighs the semantic relevance of the continuation to the given context by contrasting it with random continuations from the corpus.
4. Lexical Style Model: By focusing on the distribution of lexical items, this model ensures diversity in word choice, enhancing the style without veering off-topic.

The scoring functions of these models are integrated into a single objective using a Product of Experts (PoE) paradigm. Generation is driven by a beam search process, optimized towards this composite objective.

Experimental Results

The framework demonstrates an empirical superiority over several baselines, including adaptive softmax LLMs and recent GAN-based approaches for text generation. Notably, the human evaluations reveal a substantial preference for text generated by this framework across key communicative dimensions: coherence, clarity, relevance, and style. The system's ability to generate text that adheres more closely to the principles of effective communication marks a distinct advancement over traditional RNN outputs.

Implications and Future Directions

The introduction of cooperative discriminators opens a promising avenue for enhancing machine-generated text, particularly in contexts requiring nuanced understanding and generation of human-like discourse. This framework highlights the efficacy of intertwining generative and discriminative learning. The paper suggests future exploration into more sophisticated interplays between discriminative feedback and generator adjustments, potentially incorporating attention mechanisms or hierarchical RNN structures to better capture long-range dependencies.

As the field of AI continues exploring the frontiers of language understanding and generation, frameworks like this can play a pivotal role in advancing AI's ability to produce high-quality, coherent narratives that align closely with human communicative intents and expectations.