Efficient Machine Translation Corpus Generation
Abstract: This paper proposes an efficient and semi-automated method for human-in-the-loop post-editing for machine translation (MT) corpus generation. The method is based on online training of a custom MT quality estimation metric on-the-fly as linguists perform post-edits. The online estimator is used to prioritize worse hypotheses for post-editing, and auto-close best hypotheses without post-editing. This way, significant improvements can be achieved in the resulting quality of post-edits at a lower cost due to reduced human involvement. The trained estimator can also provide an online sanity check mechanism for post-edits and remove the need for additional linguists to review them or work on the same hypotheses. In this paper, the effect of prioritizing with the proposed method on the resulting MT corpus quality is presented versus scheduling hypotheses randomly. As demonstrated by experiments, the proposed method improves the lifecycle of MT models by focusing the linguist effort on production samples and hypotheses, which matter most for expanding MT corpora to be used for re-training them.
- Experts, errors, and context: A large-scale study of human evaluation for machine translation. Transactions of the Association for Computational Linguistics, 9:1460–1474.
- Results of the wmt21 metrics shared task: Evaluating metrics with expert-based human evaluations on ted and news domain. In Proceedings of the Sixth Conference on Machine Translation, pages 733–774.
- cushlepor: customising hlepor metric using optuna for higher agreement with human judgments or pre-trained language model labse. In Proceedings of the Sixth Conference on Machine Translation, pages 1014–1023.
- Phrase-level active learning for neural machine translation. arXiv preprint arXiv:2106.11375.
- Pushing on text readability assessment: A transformer meets handcrafted linguistic features. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10669–10686, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
- Results of the wmt20 metrics shared task. In Proceedings of the Fifth Conference on Machine Translation, pages 688–725.
- Active learning for interactive neural machine translation of data streams. arXiv preprint arXiv:1807.11243.
- Stanza: A Python natural language processing toolkit for many human languages. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations.
- Comet: A neural framework for mt evaluation. arXiv preprint arXiv:2009.09025.
- Specia, L. (2017). QT21 data. LINDAT/CLARIAH-CZ digital library at the Institute of Formal and Applied Linguistics (ÚFAL), Faculty of Mathematics and Physics, Charles University.
- Regressive ensemble for machine translation quality evaluation. arXiv preprint arXiv:2109.07242.
- Flaml: a fast and lightweight automl library. Proceedings of Machine Learning and Systems, 3:434–447.
- Empirical evaluation of active learning techniques for neural mt. In Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo 2019), pages 84–93.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.