Transfer Learning for Low-Resource Neural Machine Translation (1604.02201v1)

Abstract: The encoder-decoder framework for neural machine translation (NMT) has been shown effective in large data scenarios, but is much less effective for low-resource languages. We present a transfer learning method that significantly improves Bleu scores across a range of low-resource languages. Our key idea is to first train a high-resource language pair (the parent model), then transfer some of the learned parameters to the low-resource pair (the child model) to initialize and constrain training. Using our transfer learning method we improve baseline NMT models by an average of 5.6 Bleu on four low-resource language pairs. Ensembling and unknown word replacement add another 2 Bleu which brings the NMT performance on low-resource machine translation close to a strong syntax based machine translation (SBMT) system, exceeding its performance on one language pair. Additionally, using the transfer learning model for re-scoring, we can improve the SBMT system by an average of 1.3 Bleu, improving the state-of-the-art on low-resource machine translation.

Overview of Paper on ACL 2016 Submission Guidelines

This paper provides comprehensive guidelines for authors preparing manuscripts for submission to the ACL 2016 conference. The instructions encompass formatting requirements, submission protocols, and review processes. This document itself adheres to the specifications it delineates, serving as a practical example for authors to emulate.

Abstract and Introduction

The abstract of this paper succinctly outlines the purpose of the document: to instruct authors on preparing submissions for ACL 2016. The introduction further emphasizes the necessity for conformity to these guidelines and specifies that a Portable Document Format (PDF) is required for submissions. The introduction also addresses potential concerns for authors from countries with limited access to word-processing systems.

Submission and Camera-Ready Differences

The paper delineates the differences between initial submission and camera-ready versions. Key elements required for submission but omitted in the final camera-ready version include:

• Author-identifying information
• A ruler on the left and right margins
• Page numbers
• A confidentiality header

The paper advises authors to use the provided \LaTeX\ style files to adhere to these requirements effectively.

General Formatting Instructions

Authors must format their manuscripts in a two-column layout, except for the title, author names, affiliations, and full-width figures or tables. The maximum length is specified as eight pages for the main content, with an additional two pages allocated for references.

Instructions also specify the font sizes and styles for various sections, emphasizing the use of Adobe's Times Roman font for uniformity.

Specific Sections

The Ruler

A ruler is defined for use in the version submitted for review, facilitating reviewers in providing precise feedback. However, the camera-ready version should omit this ruler.

Electronic Resources and PDF Requirements

The document specifies the electronic resources available, including \LaTeX\ files and style sheets. Authors are required to produce their manuscripts in PDF format, ensuring all necessary fonts are embedded. Procedures for generating PDFs from postscript files on various operating systems are detailed.

First Page Formatting

The first page's layout is described thoroughly, including title placement, author information, and the abstract. The title should be centered at the top in a 15-point bold font, followed by the author names and affiliations. The abstract should be a concise summary of no more than 200 words, centered and set apart with a 12-point bold "Abstract" heading.

Review Process and Blind Review

The review process is double-blind, meaning author names and affiliations must not be included in the submitted manuscript. Self-references revealing the author's identity must be avoided. Papers not adhering to these anonymity guidelines will be rejected without review.

Multiple Submissions and Overlap Policy

Authors are required to disclose any concurrent submissions to other conferences or publications at the time of submission. The paper outlines the acceptable overlap between submissions and stipulates that submissions to ACL should not exceed 25% overlap in content with each other.

Footnotes and Graphics

Footnotes should be placed at the bottom of the page, separated by a line, and printed in a 9-point font. Graphics, including figures and tables, should be included near their first mention and must be understandable in grayscale to accommodate color-blind readers and those printing in black and white.

Implications and Future Developments

The rigor and specificity of these guidelines aim to standardize submissions, facilitating a smoother review process and promoting high-quality manuscripts. Practically, these guidelines ensure clarity and uniformity across submissions, aiding both reviewers and readers. Theoretically, adherence to these standards can improve the scientific communication process within the ACL community.

Conclusion

This document serves as an essential resource for authors intending to submit papers to ACL 2016. By providing detailed instructions and examples, it helps authors meet the conference's stringent standards, thereby maintaining the quality and integrity of the conference proceedings. Future developments may further refine these guidelines, particularly as new technologies and formats emerge within the field of computational linguistics.