Quality Estimation of Machine Translated Texts based on Direct Evidence from Training Data (2306.15399v1)
Abstract: Current Machine Translation systems achieve very good results on a growing variety of language pairs and data sets. However, it is now well known that they produce fluent translation outputs that often can contain important meaning errors. Quality Estimation task deals with the estimation of quality of translations produced by a Machine Translation system without depending on Reference Translations. A number of approaches have been suggested over the years. In this paper we show that the parallel corpus used as training data for training the MT system holds direct clues for estimating the quality of translations produced by the MT system. Our experiments show that this simple and direct method holds promise for quality estimation of translations produced by any purely data driven machine translation system.
- Prequel: Quality estimation of machine translation outputs in advance. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 11170–11183, Dec 2022.
- The IIT Bombay Hindi-English translation system at WMT 2014. In Proceedings of the Ninth Workshop on Statistical Machine Translation, pages 90–96, Baltimore, Maryland, USA, Jun 2014. Association for Computational Linguistics.
- Measuring uncertainty in translation quality evaluation (TQE). In Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 1454–1461, Marseille, Jun 2022.
- Moses: Open source toolkit for statistical machine translation. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics - Companion Volume Proceedings of the Demo and Poster Sessions, pages 177–180, Prague, Czech Republic, Jun 2007. Association for Computational Linguistics.
- Benoit Mandelbrot. Information theory and psycholinguistics. In B B Wolman and E Nagel, editors, Scientific psychology. Basic Books, 1965.
- Machine translation evaluation: Manual vs. automatic - a comparative study. In K Srujan Raju, Roman Senkerik, Satya Prasad Lanka, and V Rajagopal, editors, Data Engineering and Communication Technology, volume 1079 of Advances in Intelligent Systems and Computing, pages 541–553. Springer, 2020.
- Samanantar: The largest publicly available parallel corpora collection for 11 indic languages. arXiv cs.CL 2104.05596, 2021.
- Estimation vs metrics: is QE useful for MT model selection? In Proceedings of the 22nd Annual Conference of the European Association for Machine Translation, pages 339–346, Lisboa, Portugal, Nov 2020. European Association for Machine Translation.
- Findings of the WMT 2022 shared task on quality estimation. In Proceedings of the Seventh Conference on Machine Translation (WMT), pages 69–99, Abu Dhabi, United Arab Emirates (Hybrid), Dec 2022. Association for Computational Linguistics.
- G K Zipf. Human Behaviour and the Principle of Least effort: An introduction to Human Ecology. Addison-Wesley, 1949.