Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
80 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond (2204.09269v2)

Published 20 Apr 2022 in cs.CL and cs.LG

Abstract: Non-autoregressive (NAR) generation, which is first proposed in neural machine translation (NMT) to speed up inference, has attracted much attention in both machine learning and natural language processing communities. While NAR generation can significantly accelerate inference speed for machine translation, the speedup comes at the cost of sacrificed translation accuracy compared to its counterpart, autoregressive (AR) generation. In recent years, many new models and algorithms have been designed/proposed to bridge the accuracy gap between NAR generation and AR generation. In this paper, we conduct a systematic survey with comparisons and discussions of various non-autoregressive translation (NAT) models from different aspects. Specifically, we categorize the efforts of NAT into several groups, including data manipulation, modeling methods, training criterion, decoding algorithms, and the benefit from pre-trained models. Furthermore, we briefly review other applications of NAR models beyond machine translation, such as grammatical error correction, text summarization, text style transfer, dialogue, semantic parsing, automatic speech recognition, and so on. In addition, we also discuss potential directions for future exploration, including releasing the dependency of KD, reasonable training objectives, pre-training for NAR, and wider applications, etc. We hope this survey can help researchers capture the latest progress in NAR generation, inspire the design of advanced NAR models and algorithms, and enable industry practitioners to choose appropriate solutions for their applications. The web page of this survey is at \url{https://github.com/LitterBrother-Xiao/Overview-of-Non-autoregressive-Applications}.

PDF HTML Abstract

Overview of Non-Autoregressive Generation for Neural Machine Translation and Beyond

The paper "A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond" provides a comprehensive evaluation of non-autoregressive (NAR) methods in various natural language processing tasks, mainly focusing on neural machine translation (NMT). First introduced to accelerate inference in NMT, NAR models offer a significant speed advantage over their autoregressive (AR) counterparts at the expense of translation accuracy. Over recent years, numerous approaches have been developed to address this accuracy deficit, detailing a complex landscape of evolving NAR architectures and algorithms.

The survey dissects the field into distinct categories capturing the diversity of NAR approaches: data manipulation, modeling, training criterion, decoding algorithms, and influences of pre-trained models. For data manipulation, knowledge distillation stands out as a prevalent method to reduce data complexity, alongside innovative data learning strategies that enhance model adaptation to the training dataset. This highlights a strategic alignment between the complexity of distilled datasets and model capacity.

In modeling, the survey identifies two primary frameworks: iteration-based methods, enhancing translation quality through multiple decoding iterations, and latent variable-based methods, which leverage the probabilistic underpinnings of target-side prediction. Other enhancements aim directly at improving both the input, output, and intermediate states of the model, addressing the principal challenge of capturing target-side dependency.

Training criterion innovations, such as Connectionist Temporal Classification (CTC), N-gram-based, and order-based loss functions, have emerged as strategic solutions targeting the unique challenges posed by NAR methods, including translation coherence and variability.

Decoding remains a pivotal theme, where strategies have evolved from predictive length estimation to various innovative maneuvers like semi-autoregressive, insertion-deletion, and masked prediction decodings, balancing translation speed and accuracy.

Furthermore, leveraging pre-trained models, especially from AR methods and large-scale LLMs, provides a promising scaffold to bolster NAR performance, potentially reaching and surpassing benchmarks set by AR methods concerning both speed and accuracy.

The paper also extends the discussion to NAR adaptations in a broader spectrum, such as speech recognition, text summarization, and various other forms of automatic content generation. Each application underscores the universal challenge of missing target-side dependencies while custom tailoring existing methodologies for specialized tasks.

In conclusion, the synthesis of state-of-the-art non-autoregressive frameworks presented in this paper underscores the substantial surge in performance optimization techniques applicable across the NLP domain. Future directions are ripe for exploration in more domain-agnostic adaptations, reducing reliance on knowledge distillation, and integrating pre-training paradigms to unlock new efficiencies and performance benchmarks in real-world applications of NMT and other NLP tasks. As such, the survey stands as an essential resource for researchers endeavoring to advance NAR methodologies beyond current achievements.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (235)
  1. Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” nature, vol. 521, no. 7553, pp. 436–444, 2015.
  2. D. Bahdanau, K. H. Cho, and Y. Bengio, “Neural machine translation by jointly learning to align and translate,” in ICLR, 2015.
  3. K. Cho, B. van Merriënboer, Ç. Gu̇lçehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio, “Learning phrase representations using rnn encoder–decoder for statistical machine translation,” in EMNLP, 2014, pp. 1724–1734.
  4. I. Sutskever, O. Vinyals, and Q. V. Le, “Sequence to sequence learning with neural networks,” in NIPS, 2014, pp. 3104–3112.
  5. Y. Wu, M. Schuster, Z. Chen, Q. V. Le, M. Norouzi, W. Macherey, M. Krikun, Y. Cao, Q. Gao, K. Macherey et al., “Google’s neural machine translation system: Bridging the gap between human and machine translation,” arXiv:1609.08144, 2016.
  6. M.-T. Luong, H. Pham, and C. D. Manning, “Effective approaches to attention-based neural machine translation,” in EMNLP, 2015, pp. 1412–1421.
  7. R. Sennrich, B. Haddow, and A. Birch, “Neural machine translation of rare words with subword units,” in ACL, 2016, pp. 1715–1725.
  8. T. Mikolov, M. Karafiát, L. Burget, J. Cernockỳ, and S. Khudanpur, “Recurrent neural network based language model.” in Interspeech, vol. 2, no. 3, 2010, pp. 1045–1048.
  9. Y. LeCun, Y. Bengio et al., “Convolutional networks for images, speech, and time series,” The handbook of brain theory and neural networks, vol. 3361, no. 10, pp. 255–258, 1998.
  10. J. Gehring, M. Auli, D. Grangier, D. Yarats, and Y. N. Dauphin, “Convolutional sequence to sequence learning,” in ICML.   PMLR, 2017, pp. 1243–1252.
  11. Z. Lin, M. Feng, C. dos Santos, M. Yu, B. Xiang, B. Zhou, and Y. Bengio, “A structured self-attentive sentence embedding,” in ICLR, 2017.
  12. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in NIPS, 2017, pp. 5998–6008.
  13. M. Dehghani, S. Gouws, O. Vinyals, J. Uszkoreit, and L. Kaiser, “Universal transformers,” in ICLR, 2018.
  14. L. Wu, Y. Wang, Y. Xia, F. Tian, F. Gao, T. Qin, J. Lai, and T.-Y. Liu, “Depth growing for neural machine translation,” in ACL, 2019, pp. 5558–5563.
  15. J. Gu, J. Bradbury, C. Xiong, V. O. Li, and R. Socher, “Non-autoregressive neural machine translation,” in ICLR, 2018.
  16. C. Wang, J. Zhang, and H. Chen, “Semi-autoregressive neural machine translation,” in EMNLP, 2018, pp. 479–488.
  17. M. Ghazvininejad, O. Levy, Y. Liu, and L. Zettlemoyer, “Mask-predict: Parallel decoding of conditional masked language models,” in EMNLP-IJCNLP, 2019, pp. 6112–6121.
  18. W. Xu, S. Ma, D. Zhang, and M. Carpuat, “How does distilled data complexity impact the quality and confidence of non-autoregressive machine translation?” in Findings of ACL-IJCNLP, 2021, pp. 4392–4400.
  19. L. Ding, L. Wang, X. Liu, D. F. Wong, D. Tao, and Z. Tu, “Understanding and improving lexical choice in non-autoregressive translation,” in ICLR, 2020.
  20. J. Guo, M. Wang, D. Wei, H. Shang, Y. Wang, Z. Li, Z. Yu, Z. Wu, Y. Chen, C. Su et al., “Self-distillation mixup training for non-autoregressive neural machine translation,” arXiv:2112.11640, 2021.
  21. Y. Wang, F. Tian, D. He, T. Qin, C. Zhai, and T.-Y. Liu, “Non-autoregressive machine translation with auxiliary regularization,” in AAAI, vol. 33, no. 01, 2019, pp. 5377–5384.
  22. J. Guo, X. Tan, D. He, T. Qin, L. Xu, and T.-Y. Liu, “Non-autoregressive neural machine translation with enhanced decoder input,” in AAAI, vol. 33, no. 01, 2019, pp. 3723–3730.
  23. J. Guo, X. Tan, L. Xu, T. Qin, E. Chen, and T.-Y. Liu, “Fine-tuning by curriculum learning for non-autoregressive neural machine translation,” in AAAI, vol. 34, no. 05, 2020, pp. 7839–7846.
  24. Z. Sun, Z. Li, H. Wang, D. He, Z. Lin, and Z.-H. Deng, “Fast structured decoding for sequence models,” in NeurIPS, vol. 32, 2019, pp. 3016–3026.
  25. X. Geng, X. Feng, and B. Qin, “Learning to rewrite for non-autoregressive neural machine translation,” in EMNLP, 2021, pp. 3297–3308.
  26. A. Graves, S. Fernández, F. Gomez, and J. Schmidhuber, “Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks,” in ICML, 2006, pp. 369–376.
  27. J. Lee, E. Mansimov, and K. Cho, “Deterministic non-autoregressive neural sequence modeling by iterative refinement,” in EMNLP, 2018, pp. 1173–1182.
  28. W. Zhang, Y. Feng, F. Meng, D. You, and Q. Liu, “Bridging the gap between training and inference for neural machine translation,” in ACL, 2019, pp. 4334–4343.
  29. M. Ranzato, S. Chopra, M. Auli, and W. Zaremba, “Sequence level training with recurrent neural networks,” in ICLR, 2016.
  30. L. Wu, F. Tian, T. Qin, J. Lai, and T.-Y. Liu, “A study of reinforcement learning for neural machine translation,” in EMNLP, 2018, pp. 3612–3621.
  31. K. Zhang, R. Wang, X. Tan, J. Guo, Y. Ren, T. Qin, and T.-Y. Liu, “A study of syntactic multi-modality in non-autoregressive machine translation,” in NAACL-HLT, 2022, pp. 1747–1757.
  32. J. D. M.-W. C. Kenton and L. K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in NAACL-HLT, 2019, pp. 4171–4186.
  33. J. Gu, C. Wang, and J. Zhao, “Levenshtein transformer,” in NeurIPS, vol. 32, 2019, pp. 11 181–11 191.
  34. J. Kasai, J. Cross, M. Ghazvininejad, and J. Gu, “Non-autoregressive machine translation with disentangled context transformer,” in ICML.   PMLR, 2020, pp. 5144–5155.
  35. C. Tian, Y. Wang, H. Cheng, Y. Lian, and Z. Zhang, “Train once, and decode as you like,” in COLING, 2020, pp. 280–293.
  36. C. Saharia, W. Chan, S. Saxena, and M. Norouzi, “Non-autoregressive machine translation with latent alignments,” in EMNLP, 2020, pp. 1098–1108.
  37. L. Qian, H. Zhou, Y. Bao, M. Wang, L. Qiu, W. Zhang, Y. Yu, and L. Li, “Glancing transformer for non-autoregressive neural machine translation,” in ACL-IJCNLP, 2021, pp. 1993–2003.
  38. F. Huang, H. Zhou, Y. Liu, H. Li, and M. Huang, “Directed acyclic transformer for non-autoregressive machine translation,” in ICML.   PMLR, 2022, pp. 9410–9428.
  39. C. Du, Z. Tu, and J. Jiang, “Order-agnostic cross entropy for non-autoregressive machine translation,” in ICML.   PMLR, 2021, pp. 2849–2859.
  40. P. Li, L. Li, M. Zhang, M. Wu, and Q. Liu, “Universal conditional masked language pre-training for neural machine translation,” in ACL, 2022, pp. 6379–6391.
  41. G. Hinton, O. Vinyals, J. Dean et al., “Distilling the knowledge in a neural network,” arXiv:1503.02531, vol. 2, no. 7, 2015.
  42. Y. Kim and A. M. Rush, “Sequence-level knowledge distillation,” in EMNLP, 2016, pp. 1317–1327.
  43. C. Zhou, J. Gu, and G. Neubig, “Understanding knowledge distillation in non-autoregressive machine translation,” in ICLR, 2019.
  44. Y. Ren, J. Liu, X. Tan, Z. Zhao, S. Zhao, and T.-Y. Liu, “A study of non-autoregressive model for sequence generation,” in ACL, 2020, pp. 149–159.
  45. T. Furlanello, Z. Lipton, M. Tschannen, L. Itti, and A. Anandkumar, “Born again neural networks,” in ICML.   PMLR, 2018, pp. 1607–1616.
  46. T. Shen, M. Ott, M. Auli, and M. Ranzato, “Mixture models for diverse machine translation: Tricks of the trade,” in ICML.   PMLR, 2019, pp. 5719–5728.
  47. J. Lee, D. Tran, O. Firat, and K. Cho, “On the discrepancy between density estimation and sequence generation,” in Proceedings of the Fourth Workshop on Structured Prediction for NLP, 2020, pp. 84–94.
  48. L. Ding, L. Wang, X. Liu, D. F. Wong, D. Tao, and z. Tu, “Rejuvenating low-frequency words: Making the most of parallel data in non-autoregressive translation,” in ACL-IJCNLP, 2021, pp. 3431–3441.
  49. J. Zhou and P. Keung, “Improving non-autoregressive neural machine translation with monolingual data,” in ACL, 2020, pp. 1893–1898.
  50. C. Shao, X. Wu, and Y. Feng, “One reference is not enough: Diverse distillation with reference selection for non-autoregressive translation,” in NAACL-HLT, 2022, pp. 3779–3791.
  51. Y. Bao, H. Zhou, S. Huang, D. Wang, L. Qian, X. Dai, J. Chen, and L. Li, “latent-glat: Glancing at latent variables for parallel text generation,” in ACL, 2022, pp. 8398–8409.
  52. L. Ding, L. Wang, X. Liu, D. F. Wong, D. Tao, and Z. Tu, “Progressive multi-granularity training for non-autoregressive translation,” in Findings of ACL-IJCNLP, 2021, pp. 2797–2803.
  53. P. Xie, Z. Li, Z. Zhao, J. Liu, and X. Hu, “Mvsr-nat: Multi-view subset regularization for non-autoregressive machine translation,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, pp. 1–10, 2022.
  54. M. Zhu, J. Wang, and C. Yan, “Non-autoregressive neural machine translation with consistency regularization optimized variational framework,” in NAACL-HLT, 2022, pp. 607–617.
  55. H. Cheng and Z. Zhang, “Con-nat: Contrastive non-autoregressive neural machine translation,” in Findings of EMNLP, 2022, pp. 6219–6231.
  56. M. Stern, W. Chan, J. Kiros, and J. Uszkoreit, “Insertion transformer: Flexible sequence generation via insertion operations,” in ICML.   PMLR, 2019, pp. 5976–5985.
  57. M. Ghazvininejad, O. Levy, and L. Zettlemoyer, “Semi-autoregressive training improves mask-predict decoding,” arXiv:2001.08785, 2020.
  58. J. Guo, L. Xu, and E. Chen, “Jointly masked sequence-to-sequence model for non-autoregressive neural machine translation,” in ACL, 2020, pp. 376–385.
  59. X. S. Huang, F. Perez, and M. Volkovs, “Improving non-autoregressive translation models without distillation,” in ICLR, 2022.
  60. N. Savinov, J. Chung, M. Binkowski, E. Elsen, and A. van den Oord, “Step-unrolled denoising autoencoders for text generation,” in ICLR, 2023.
  61. S. Lu, T. Meng, and N. Peng, “Insnet: An efficient, flexible, and performant insertion-based text generation model,” in NeurIPS, vol. 35, 2022, pp. 7011–7023.
  62. X. Ma, C. Zhou, X. Li, G. Neubig, and E. Hovy, “Flowseq: Non-autoregressive conditional sequence generation with generative flow,” in EMNLP-IJCNLP, 2019, pp. 4282–4292.
  63. Y. Bao, H. Zhou, J. Feng, M. Wang, S. Huang, J. Chen, and L. Li, “Non-autoregressive transformer by position learning,” arXiv:1911.10677, 2019.
  64. N. Akoury, K. Krishna, and M. Iyyer, “Syntactically supervised transformers for faster neural machine translation,” in ACL, 2019, pp. 1269–1281.
  65. R. Shu, J. Lee, H. Nakayama, and K. Cho, “Latent-variable non-autoregressive neural machine translation with deterministic inference using a delta posterior,” in AAAI, vol. 34, no. 05, 2020, pp. 8846–8853.
  66. Q. Ran, Y. Lin, P. Li, and J. Zhou, “Guiding non-autoregressive neural machine translation decoding with reordering information,” in AAAI, vol. 35, no. 15, 2021, pp. 13 727–13 735.
  67. J. Song, S. Kim, and S. Yoon, “Alignart: Non-autoregressive neural machine translation by jointly learning to estimate alignment and translate,” in EMNLP, 2021, pp. 1–14.
  68. Y. Bao, S. Huang, T. Xiao, D. Wang, X. Dai, and J. Chen, “Non-autoregressive translation by learning target categorical codes,” in NAACL-HLT, 2021, pp. 5749–5759.
  69. Y. Liu, Y. Wan, J. Zhang, W. Zhao, and S. Y. Philip, “Enriching non-autoregressive transformer with syntactic and semantic structures for neural machine translation,” in EACL, 2021, pp. 1235–1244.
  70. J. Gu and X. Kong, “Fully non-autoregressive neural machine translation: Tricks of the trade,” in Findings of ACL-IJCNLP, 2021, pp. 120–133.
  71. X. Li, Y. Meng, A. Yuan, F. Wu, and J. Li, “Lava nat: A non-autoregressive translation model with look-around decoding and vocabulary attention,” arXiv:2002.03084, 2020.
  72. L. Ding, L. Wang, D. Wu, D. Tao, and Z. Tu, “Context-aware cross-attention for non-autoregressive translation,” in COLING, 2020, pp. 4396–4402.
  73. C. Huang, H. Zhou, O. R. Zaïane, L. Mou, and L. Li, “Non-autoregressive translation with layer-wise prediction and deep supervision,” in AAAI, vol. 36, no. 10, 2022, pp. 10 776–10 784.
  74. J. Zhan, Q. Chen, B. Chen, W. Wang, Y. Bai, and Y. Gao, “Non-autoregressive translation with dependency-aware decoder,” arXiv:2203.16266, 2022.
  75. C. Shao, Z. Ma, and Y. Feng, “Viterbi decoding of directed acyclic transformer for non-autoregressive machine translation,” in Findings of EMNLP, 2022, pp. 4390–4397.
  76. Z. Ma, C. Shao, S. Gui, M. Zhang, and Y. Feng, “Fuzzy alignments in directed acyclic graph for non-autoregressive machine translation,” in ICLR, 2023.
  77. C. Shao, J. Zhang, Y. Feng, F. Meng, and J. Zhou, “Minimizing the bag-of-ngrams difference for non-autoregressive neural machine translation,” in AAAI, vol. 34, no. 01, 2020, pp. 198–205.
  78. G. Marjan, V. Karpukhin, L. Zettlemoyer, and O. Levy, “Aligned cross entropy for non-autoregressive machine translation,” in ICML.   PMLR, 2020, pp. 3515–3523.
  79. G. Liu, Z. Yang, T. Tao, X. Liang, J. Bao, Z. Li, X. He, S. Cui, and Z. Hu, “Don’t take it literally: An edit-invariant sequence loss for text generation,” in NAACL-HLT, 2022, pp. 2055–2078.
  80. C. Du, Z. Tu, L. Wang, and J. Jiang, “ngram-oaxe: Phrase-based order-agnostic cross entropy for non-autoregressive machine translation,” in COLING, 2022, pp. 5035–5045.
  81. Y. Li, L. Cui, Y. Yin, and Y. Zhang, “Multi-granularity optimization for non-autoregressive translation,” arXiv:2210.11017, 2022.
  82. C. Shao and Y. Feng, “Non-monotonic latent alignments for ctc-based non-autoregressive machine translation,” in NeurIPS, vol. 35, 2022, pp. 8159–8173.
  83. Q. Ran, Y. Lin, P. Li, and J. Zhou, “Learning to recover from multi-modality errors for non-autoregressive neural machine translation,” in ACL, 2020, pp. 3059–3069.
  84. H. Xia, T. Ge, F. Wei, and Z. Sui, “Lossless speedup of autoregressive translation with generalized aggressive decoding,” arXiv:2203.16487, 2022.
  85. W. Chan, N. Kitaev, K. Guu, M. Stern, and J. Uszkoreit, “Kermit: Generative insertion-based modeling for sequences,” arXiv:1906.01604, 2019.
  86. M. Wang, J. Guo, Y. Wang, D. Wei, H. Shang, Y. Li, C. Su, Y. Chen, M. Zhang, S. Tao et al., “Diformer: Directional transformer for neural machine translation,” in Proceedings of the 23rd Annual Conference of the European Association for Machine Translation, 2022, pp. 81–90.
  87. Q. Wang, X. Hu, and M. Chen, “Hybrid-regressive neural machine translation,” arXiv:2210.10416, 2022.
  88. B. Wei, M. Wang, H. Zhou, J. Lin, and X. Sun, “Imitation learning for non-autoregressive neural machine translation,” in ACL, 2019, pp. 1304–1312.
  89. Z. Li, Z. Lin, D. He, F. Tian, T. Qin, L. Wang, and T.-Y. Liu, “Hint-based training for non-autoregressive machine translation,” in EMNLP-IJCNLP, 2019, pp. 5708–5713.
  90. L. Tu, R. Y. Pang, S. Wiseman, and K. Gimpel, “Engine: Energy-based inference networks for non-autoregressive machine translation,” in ACL, 2020, pp. 2819–2826.
  91. Z. Sun and Y. Yang, “An em approach to non-autoregressive conditional sequence generation,” in ICML.   PMLR, 2020, pp. 9249–9258.
  92. Y. Hao, S. He, W. Jiao, Z. Tu, M. Lyu, and X. Wang, “Multi-task learning with shared encoder for non-autoregressive machine translation,” in NAACL-HLT, 2021, pp. 3989–3996.
  93. J. Liu, Y. Ren, X. Tan, C. Zhang, T. Qin, Z. Zhao, and T.-Y. Liu, “Task-level curriculum learning for non-autoregressive neural machine translation,” in IJCAI, 2021, pp. 3861–3867.
  94. X. Wang, Z. Zheng, and S. Huang, “Helping the weak makes you strong: Simple multi-task learning improves non-autoregressive translators,” in EMNLP, 2022, pp. 5513–5519.
  95. J. Guo, Z. Zhang, L. Xu, H.-R. Wei, B. Chen, and E. Chen, “Incorporating bert into parallel sequence decoding with adapters,” in NeurIPS, vol. 33, 2020, pp. 10 843–10 854.
  96. Y. Su, D. Cai, Y. Wang, D. Vandyke, S. Baker, P. Li, and N. Collier, “Non-autoregressive text generation with pre-trained language models,” in EACL, 2021, pp. 234–243.
  97. Y. Wang, S. He, G. Chen, Y. Chen, and D. Jiang, “Xlm-d: Decorate cross-lingual pre-training model as non-autoregressive neural machine translation,” in EMNLP, 2022, pp. 6934–6946.
  98. Y. Bengio, J. Louradour, R. Collobert, and J. Weston, “Curriculum learning,” in ICML, 2009, pp. 41–48.
  99. Z. Song, H. Zhou, L. Qian, J. Xu, S. Cheng, M. Wang, and L. Li, “switch-GLAT: Multilingual parallel machine translation via code-switch decoder,” in ICLR, 2022.
  100. X. Liang, L. Wu, J. Li, Y. Wang, Q. Meng, T. Qin, W. Chen, M. Zhang, T.-Y. Liu et al., “R-drop: regularized dropout for neural networks,” in NeurIPS, vol. 34, 2021, pp. 10 890–10 905.
  101. S. Welleck, K. Brantley, H. D. Iii, and K. Cho, “Non-monotonic sequential text generation,” in ICML.   PMLR, 2019, pp. 6716–6726.
  102. J. Gu, Q. Liu, and K. Cho, “Insertion-based decoding with automatically inferred generation order,” Transactions of the Association for Computational Linguistics, vol. 7, pp. 661–676, 2019.
  103. J. Xu, J. Crego, and F. Yvon, “Bilingual synchronization: Restoring translational relationships with editing operations,” in EMNLP, 2022, pp. 8016–8030.
  104. ——, “Non-autoregressive machine translation with translation memories,” arXiv preprint arXiv:2210.06020, 2022.
  105. A. Niwa, T. Sho, and O. Naoaki, “Nearest neighbor non-autoregressive text generation,” arXiv:2208.12496, 2022.
  106. P. Xie, Z. Cui, X. Chen, X. Hu, J. Cui, and B. Wang, “Infusing sequential information into conditional masked translation model with self-review mechanism,” in COLING, 2020, pp. 15–25.
  107. L. Kaiser, S. Bengio, A. Roy, A. Vaswani, N. Parmar, J. Uszkoreit, and N. Shazeer, “Fast decoding in sequence models using discrete latent variables,” in ICML.   PMLR, 2018, pp. 2390–2399.
  108. C. Shao, J. Zhang, J. Zhou, and Y. Feng, “Rephrasing the reference for non-autoregressive machine translation,” arXiv preprint arXiv:2211.16863, 2022.
  109. J. Libovickỳ and J. Helcl, “End-to-end non-autoregressive neural machine translation with connectionist temporal classification,” in EMNLP, 2018, pp. 3016–3021.
  110. M. Wang, G. Jiaxin, Y. Wang, Y. Chen, S. Chang, H. Shang, M. Zhang, S. Tao, and H. Yang, “How length prediction influence the performance of non-autoregressive translation?” in Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, 2021, pp. 205–213.
  111. W. Chan, M. Stern, J. Kiros, and J. Uszkoreit, “An empirical study of generation order for machine translation,” in EMNLP, 2020, pp. 5764–5773.
  112. J. Kreutzer, G. Foster, and C. Cherry, “Inference strategies for machine translation with conditional masking,” arXiv:2010.02352, 2020.
  113. Z. Kasner, J. Libovickỳ, and J. Helcl, “Improving fluency of non-autoregressive machine translation,” arXiv:2004.03227, 2020.
  114. J. Zhu, Y. Xia, L. Wu, D. He, T. Qin, W. Zhou, H. Li, and T. Liu, “Incorporating bert into neural machine translation,” in ICLR, 2020.
  115. Z. Yang, B. Hu, A. Han, S. Huang, and Q. Ju, “Csp: Code-switching pre-training for neural machine translation,” in EMNLP, 2020, pp. 2624–2636.
  116. J. Helcl, B. Haddow, and A. Birch, “Non-autoregressive machine translation: It’s not as fast as it seems,” in NAACL-HLT, 2022, pp. 1780–1790.
  117. J. Kasai, N. Pappas, H. Peng, J. Cross, and N. Smith, “Deep encoder, shallow decoder: Reevaluating non-autoregressive machine translation,” in ICLR, 2020.
  118. K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a method for automatic evaluation of machine translation,” in ACL, 2002, pp. 311–318.
  119. R. Schmidt, T. Pires, S. Peitz, and J. Lööf, “Non-autoregressive neural machine translation: A call for clarity,” in EMNLP, 2022, pp. 2785–2799.
  120. M. Popović, “chrf: character n-gram f-score for automatic mt evaluation,” in the Tenth Workshop on SMT, 2015, pp. 392–395.
  121. M. Post, “A call for clarity in reporting bleu scores,” in Proceedings of the Third Conference on Machine Translation: Research Papers, 2018, pp. 186–191.
  122. R. Rei, C. Stewart, A. C. Farinha, and A. Lavie, “Comet: A neural framework for mt evaluation,” in EMNLP), 2020, pp. 2685–2702.
  123. A. Babu, A. Shrivastava, A. Aghajanyan, A. Aly, A. Fan, and M. Ghazvininejad, “Non-autoregressive semantic parsing for compositional task-oriented dialog,” in NAACL-HLT, 2021, pp. 2969–2978.
  124. A. Shrivastava, P. Chuang, A. Babu, S. Desai, A. Arora, A. Zotov, and A. Aly, “Span pointer networks for non-autoregressive task-oriented semantic parsing,” in Findings of EMNLP, 2021, pp. 1873–1886.
  125. Y. Ren, Y. Ruan, X. Tan, T. Qin, S. Zhao, Z. Zhao, and T.-Y. Liu, “Fastspeech: Fast, robust and controllable text to speech,” in NeurIPS, vol. 32, 2019.
  126. K. Peng, W. Ping, Z. Song, and K. Zhao, “Non-autoregressive neural text-to-speech,” in ICML.   PMLR, 2020, pp. 7586–7598.
  127. K. Yang, W. Lei, D. Liu, W. Qi, and J. Lv, “Pos-constrained parallel decoding for non-autoregressive generation,” in ACL, 2021, pp. 5990–6000.
  128. T. Jiang, S. Huang, Z. Zhang, D. Wang, F. Zhuang, F. Wei, H. Huang, L. Zhang, and Q. Zhang, “Improving non-autoregressive generation with mixup training,” arXiv:2110.11115, 2021.
  129. Y. Higuchi, N. Chen, Y. Fujita, H. Inaguma, T. Komatsu, J. Lee, J. Nozaki, T. Wang, and S. Watanabe, “A comparative study on non-autoregressive modelings for speech-to-text generation,” in 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU).   IEEE, 2021, pp. 47–54.
  130. F.-H. Yu and K.-Y. Chen, “Non-autoregressive transformer-based end-to-end asr using bert,” arXiv:2104.04805, 2021.
  131. X. Song, Z. Wu, Y. Huang, C. Weng, D. Su, and H. Meng, “Non-autoregressive transformer asr with ctc-enhanced decoder input,” in ICASSP.   IEEE, 2021, pp. 5894–5898.
  132. W. Qi, Y. Gong, J. Jiao, Y. Yan, W. Chen, D. Liu, K. Tang, H. Li, J. Chen, R. Zhang et al., “Bang: Bridging autoregressive and non-autoregressive generation with large scale pretraining,” in ICML.   PMLR, 2021, pp. 8630–8639.
  133. M. Straka, J. Náplava, and J. Straková, “Character transformations for non-autoregressive gec tagging,” in W-NUT, 2021, pp. 417–422.
  134. P. Li and S. Shi, “Tail-to-tail non-autoregressive sequence prediction for chinese grammatical error correction,” in ACL-IJCNLP, 2021, pp. 4973–4984.
  135. Q. Han, Y. Meng, F. Wu, and J. Li, “Non-autoregressive neural dialogue generation,” arXiv:2002.04250, 2020.
  136. Y. Zou, Z. Liu, X. Hu, and Q. Zhang, “Thinking clearly, talking fast: Concept-guided non-autoregressive generation for open-domain dialogue systems,” in EMNLP, 2021, pp. 2215–2226.
  137. J. Mallinson, J. Adamek, E. Malmi, and A. Severyn, “Edit5: Semi-autoregressive text-editing with t5 warm-start,” arXiv:2205.12209, 2022.
  138. W. Qi, Y. Gong, Y. Shen, J. Jiao, Y. Yan, H. Li, R. Zhang, W. Chen, and N. Duan, “A self-paced mixed distillation method for non-autoregressive generation,” arXiv:2205.11162, 2022.
  139. S. Agrawal and M. Carpuat, “An imitation learning curriculum for text editing with non-autoregressive models,” in ACL, 2022, pp. 7550–7563.
  140. J. Li, T. Tang, W. X. Zhao, J.-Y. Nie, and J.-R. Wen, “ELMER: A non-autoregressive pre-trained language model for efficient and effective text generation,” in EMNLP, 2022, pp. 1044–1058.
  141. E. A. Chi, J. Salazar, and K. Kirchhoff, “Align-refine: Non-autoregressive speech recognition via iterative realignment,” in NAACL-HLT, 2021, pp. 1920–1927.
  142. N. Chen, P. Zelasko, L. Moro-Velázquez, J. Villalba, and N. Dehak, “Align-denoise: Single-pass non-autoregressive speech recognition,” in Interspeech, 2021, pp. 3770–3774.
  143. N. Chen, S. Watanabe, J. Villalba, P. Żelasko, and N. Dehak, “Non-autoregressive transformer for speech recognition,” IEEE Signal Processing Letters, vol. 28, pp. 121–125, 2020.
  144. W. Chan, C. Saharia, G. Hinton, M. Norouzi, and N. Jaitly, “Imputer: Sequence modelling via imputation and dynamic programming,” in ICML.   PMLR, 2020, pp. 1403–1413.
  145. Y. Higuchi, S. Watanabe, N. Chen, T. Ogawa, and T. Kobayashi, “Mask ctc: Non-autoregressive end-to-end asr with ctc and mask predict,” in Interspeech, 2020, pp. 3655–3659.
  146. X. Gong, Z. Zhou, and Y. Qian, “Knowledge transfer and distillation from autoregressive to non-autoregressive speech recognition,” arXiv:2207.10600, 2022.
  147. J. W. Yoon, B. J. Woo, S. Ahn, H. Lee, and N. S. Kim, “Inter-kd: Intermediate knowledge distillation for ctc-based automatic speech recognition,” in SLT.   IEEE, 2023, pp. 280–286.
  148. J. Lee and S. Watanabe, “Intermediate loss regularization for ctc-based speech recognition,” in ICASSP.   IEEE, 2021, pp. 6224–6228.
  149. J. Nozaki and T. Komatsu, “Relaxing the conditional independence assumption of ctc-based asr by conditioning on intermediate predictions,” arXiv:2104.02724, 2021.
  150. K. Deng, Z. Yang, S. Watanabe, Y. Higuchi, G. Cheng, and P. Zhang, “Improving non-autoregressive end-to-end speech recognition with pre-trained acoustic and language models,” in ICASSP.   IEEE, 2022, pp. 8522–8526.
  151. R. Fan, W. Chu, P. Chang, and J. Xiao, “Cass-nat: Ctc alignment-based single step non-autoregressive transformer for speech recognition,” in ICASSP.   IEEE, 2021, pp. 5889–5893.
  152. Y. Higuchi, H. Inaguma, S. Watanabe, T. Ogawa, and T. Kobayashi, “Improved mask-ctc for non-autoregressive end-to-end asr,” in ICASSP.   IEEE, 2021, pp. 8363–8367.
  153. Y. Yang, Y. Li, and B. Du, “Improving ctc-based asr models with gated interlayer collaboration,” arXiv:2205.12462, 2022.
  154. K.-H. Lu and K.-Y. Chen, “A context-aware knowledge transferring strategy for ctc-based asr,” in SLT.   IEEE, 2023, pp. 60–67.
  155. S. Dingliwal, M. Sunkara, S. Ronanki, J. Farris, K. Kirchhoff, and S. Bodapati, “Personalization of ctc speech recognition models,” in SLT.   IEEE, 2023, pp. 302–309.
  156. Y. Bai, J. Yi, J. Tao, Z. Tian, Z. Wen, and S. Zhang, “Fast end-to-end speech recognition via non-autoregressive models and cross-modal knowledge transferring from bert,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 1897–1911, 2021.
  157. Y. Higuchi, T. Ogawa, T. Kobayashi, and S. Watanabe, “Bectra: Transducer-based end-to-end asr with bert-enhanced encoder,” arXiv:2211.00792, 2022.
  158. R. Fan, G. Ye, Y. Gaur, and J. Li, “Acoustic-aware non-autoregressive spell correction with mask sample decoding,” arXiv:2210.08665, 2022.
  159. H. Futami, H. Inaguma, S. Ueno, M. Mimura, S. Sakai, and T. Kawahara, “Non-autoregressive error correction for ctc-based asr with phone-conditioned masked lm,” arXiv:2209.04062, 2022.
  160. C. H. Puyuan Liu and L. Mou, “Learning non-autoregressive models from search for unsupervised sentence summarization,” in ACL, 2022, pp. 7916–7929.
  161. P. Liu, X. Zhang, and L. Mou, “A character-level length-control algorithm for non-autoregressive sentence summarization,” in NeurIPS, vol. 35, 2022, pp. 29 101–29 112.
  162. K. Shen, Y. Leng, X. Tan, S. Tang, Y. Zhang, W. Liu, and E. Lin, “Mask the correct tokens: An embarrassingly simple approach for error correction,” arXiv:2211.13252, 2022.
  163. I. Kulikov, A. Miller, K. Cho, and J. Weston, “Importance of search and evaluation strategies in neural dialogue modeling,” in INLG, 2019, pp. 76–87.
  164. L. Cheng, W. Jia, and W. Yang, “An effective non-autoregressive model for spoken language understanding,” in CIKM, 2021, pp. 241–250.
  165. ——, “Capture salient historical information: A fast and accurate non-autoregressive model for multi-turn spoken language understanding,” ACM Transactions on Information Systems, vol. 41, no. 2, pp. 1–32, 2022.
  166. Y. Ma and Q. Li, “Exploring non-autoregressive text style transfer,” in EMNLP, 2021, pp. 9267–9278.
  167. F. Huang, Z. Chen, C. H. Wu, Q. Guo, X. Zhu, and M. Huang, “Nast: A non-autoregressive generator with word alignment for unsupervised text style transfer,” in Findings of ACL-IJCNLP, 2021, pp. 1577–1590.
  168. H. Zhang, H. Song, S. Li, M. Zhou, and D. Song, “A survey of controllable text generation using transformer-based pre-trained language models,” arXiv:2201.05337, 2022.
  169. S. Agrawal, W. Xu, and M. Carpuat, “A non-autoregressive edit-based approach to controllable text simplification,” in Findings of ACL-IJCNLP, 2021, pp. 3757–3769.
  170. H. Iso, “Autotemplate: A simple recipe for lexically constrained text generation,” arXiv:2211.08387, 2022.
  171. X. Li, J. Thickstun, I. Gulrajani, P. S. Liang, and T. B. Hashimoto, “Diffusion-lm improves controllable text generation,” in NeurIPS, vol. 35, 2022, pp. 4328–4343.
  172. S. Kumar, B. Paria, and Y. Tsvetkov, “Gradient-based constrained sampling from language models,” arXiv:2205.12558, 2022.
  173. Z.-c. Fei, “Fast image caption generation with position alignment,” arXiv:1912.06365, 2019.
  174. J. Gao, X. Meng, S. Wang, X. Li, S. Wang, S. Ma, and W. Gao, “Masked non-autoregressive image captioning,” arXiv:1906.00717, 2019.
  175. L. Guo, J. Liu, X. Zhu, X. He, J. Jiang, and H. Lu, “Non-autoregressive image captioning with counterfactuals-critical multi-agent learning,” in IJCAI, 2021, pp. 767–773.
  176. Z. Wang, L. Chen, W. Ma, G. Han, Y. Niu, J. Shao, and J. Xiao, “Explicit image caption editing,” in ECCV.   Springer, 2022, pp. 113–129.
  177. Z. Fei, “Efficient modeling of future context for image captioning,” in ACM MM, 2022, pp. 5026–5035.
  178. C. Zhou, F. Meng, J. Zhou, M. Zhang, H. Wang, and J. Su, “Confidence based bidirectional global context aware training framework for neural machine translation,” in ACL, 2022, pp. 2878–2889.
  179. Z. Fei, M. Fan, L. Zhu, J. Huang, X. Wei, and X. Wei, “Uncertainty-aware image captioning,” arXiv:2211.16769, 2022.
  180. Q. Chen, C. Deng, and Q. Wu, “Learning distinct and representative modes for image captioning,” in NeurIPS, vol. 35, 2022, pp. 9472–9485.
  181. T. Zhang, H. Xu, J. van Genabith, D. Xiong, and H. Zan, “Napg: Non-autoregressive program generation for hybrid tabular-textual question answering,” arXiv:2211.03462, 2022.
  182. J. Wang, C. Wang, M. Qiu, Q. Shi, H. Wang, J. Huang, and M. Gao, “Kecp: Knowledge enhanced contrastive prompting for few-shot extractive question answering,” arXiv:2205.03071, 2022.
  183. H. Lu, Z. Wu, X. Wu, X. Li, S. Kang, X. Liu, and H. Meng, “Vaenar-tts: Variational auto-encoder based non-autoregressive text-to-speech synthesis,” arXiv:2107.03298, 2021.
  184. R. Shah, K. Pokora, A. Ezzerg, V. Klimkov, G. Huybrechts, B. Putrycz, D. Korzekwa, and T. Merritt, “Non-autoregressive tts with explicit duration modelling for low-resource highly expressive speech,” arXiv:2106.12896, 2021.
  185. P. Liu, Y. Cao, S. Liu, N. Hu, G. Li, C. Weng, and D. Su, “Vara-tts: Non-autoregressive text-to-speech synthesis based on very deep vae with residual attention,” arXiv:2102.06431, 2021.
  186. Y. Lee, J. Shin, and K. Jung, “Bidirectional variational inference for non-autoregressive text-to-speech,” in ICLR, 2020.
  187. Y. Ren, X. Tan, T. Qin, Z. Zhao, and T.-Y. Liu, “Revisiting over-smoothness in text to speech,” in ACL, 2022, pp. 8197–8213.
  188. Y. Wang, R. Skerry-Ryan, D. Stanton, Y. Wu, R. J. Weiss, N. Jaitly, Z. Yang, Y. Xiao, Z. Chen, S. Bengio et al., “Tacotron: Towards end-to-end speech synthesis,” in Interspeech, 2017, pp. 4006–4010.
  189. A. Łańcucki, “Fastpitch: Parallel text-to-speech with pitch prediction,” in ICASSP.   IEEE, 2021, pp. 6588–6592.
  190. H. Guo, H. Lu, X. Wu, and H. Meng, “A multi-scale time-frequency spectrogram discriminator for gan-based non-autoregressive tts,” arXiv:2203.01080, 2022.
  191. J. Kim, S. Kim, J. Kong, and S. Yoon, “Glow-tts: A generative flow for text-to-speech via monotonic alignment search,” in NeurIPS, vol. 33, 2020, pp. 8067–8077.
  192. J. Bae, J. Yang, T. Bak, and Y.-S. Joo, “Hierarchical and Multi-Scale Variational Autoencoder for Diverse and Natural Non-Autoregressive Text-to-Speech,” in Interspeech, 2022, pp. 813–817.
  193. Y. Li, C. Yu, G. Sun, H. Jiang, F. Sun, W. Zu, Y. Wen, Y. Yang, and J. Wang, “Cross-utterance conditioned vae for non-autoregressive text-to-speech,” in ACL, 2022, pp. 391–400.
  194. Z. Liu, Q. Tian, C. Hu, X. Liu, M. Wu, Y. Wang, H. Zhao, and Y. Wang, “Controllable and lossless non-autoregressive end-to-end text-to-speech,” arXiv:2207.06088, 2022.
  195. M. Post, G. Kumar, A. Lopez, D. Karakos, C. Callison-Burch, and S. Khudanpur, “Improved speech-to-text translation with the fisher and callhome spanish-english speech translation corpus,” in SLT Workshop, 2013.
  196. S.-P. Chuang, Y.-S. Chuang, C.-C. Chang, and H.-Y. Lee, “Investigating the reordering capability in ctc-based non-autoregressive end-to-end speech translation,” in Findings of ACL-IJCNLP, 2021, pp. 1068–1077.
  197. H. Inaguma, Y. Higuchi, K. Duh, T. Kawahara, and S. Watanabe, “Orthros: Non-autoregressive end-to-end speech translation with dual-decoder,” in ICASSP.   IEEE, 2021, pp. 7503–7507.
  198. ——, “Non-autoregressive end-to-end speech translation with parallel autoregressive rescoring,” arXiv:2109.04411, 2021.
  199. S. Dalmia, B. Yan, V. Raunak, F. Metze, and S. Watanabe, “Searchable hidden intermediates for end-to-end models of decomposable sequence tasks,” in NAACL-HLT, 2021.
  200. J. Shi, J. D. Amith, X. Chang, S. Dalmia, B. Yan, and S. Watanabe, “Highland puebla nahuatl speech translation corpus for endangered language documentation,” in AmericasNLP, 2021, pp. 53–63.
  201. H. Inaguma, S. Dalmia, B. Yan, and S. Watanabe, “Fast-md: Fast multi-decoder end-to-end speech translation with non-autoregressive hidden intermediates,” in ASRU.   IEEE, 2021, pp. 922–929.
  202. J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli, “Deep unsupervised learning using nonequilibrium thermodynamics,” in ICML.   PMLR, 2015, pp. 2256–2265.
  203. J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” in NeurIPS, vol. 33, 2020, pp. 6840–6851.
  204. Z. Kong, W. Ping, J. Huang, K. Zhao, and B. Catanzaro, “Diffwave: A versatile diffusion model for audio synthesis,” in ICLR, 2020.
  205. G. Mittal, J. Engel, C. Hawthorne, and I. Simon, “Symbolic music generation with diffusion models,” arXiv:2103.16091, 2021.
  206. A. Q. Nichol and P. Dhariwal, “Improved denoising diffusion probabilistic models,” in ICML.   PMLR, 2021, pp. 8162–8171.
  207. J. Austin, D. D. Johnson, J. Ho, D. Tarlow, and R. van den Berg, “Structured denoising diffusion models in discrete state-spaces,” in NeurIPS, vol. 34, 2021, pp. 17 981–17 993.
  208. C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. L. Denton, K. Ghasemipour, R. Gontijo Lopes, B. Karagol Ayan, T. Salimans, J. Ho, D. J. Fleet, and M. Norouzi, “Photorealistic text-to-image diffusion models with deep language understanding,” in NeurIPS, vol. 35, 2022, pp. 36 479–36 494.
  209. A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen, “Hierarchical text-conditional image generation with clip latents,” arXiv:2204.06125, 2022.
  210. A. Q. Nichol, P. Dhariwal, A. Ramesh, P. Shyam, P. Mishkin, B. Mcgrew, I. Sutskever, and M. Chen, “Glide: Towards photorealistic image generation and editing with text-guided diffusion models,” in ICML.   PMLR, 2022, pp. 16 784–16 804.
  211. E. Hoogeboom, D. Nielsen, P. Jaini, P. Forré, and M. Welling, “Argmax flows and multinomial diffusion: Learning categorical distributions,” in NeurIPS, vol. 34, 2021, pp. 12 454–12 465.
  212. E. Hoogeboom, A. A. Gritsenko, J. Bastings, B. Poole, R. van den Berg, and T. Salimans, “Autoregressive diffusion models,” in ICLR, 2022.
  213. S. Gong, M. Li, J. Feng, Z. Wu, and L. Kong, “Diffuseq: Sequence to sequence text generation with diffusion models,” arXiv:2210.08933, 2022.
  214. P. Yu, S. Xie, X. Ma, B. Jia, B. Pang, R. Gao, Y. Zhu, S.-C. Zhu, and Y. N. Wu, “Latent diffusion energy-based model for interpretable text modelling,” in ICML.   PMLR, 2022, pp. 25 702–25 720.
  215. M. Reid, V. J. Hellendoorn, and G. Neubig, “Diffuser: Discrete diffusion via edit-based reconstruction,” arXiv:2210.16886, 2022.
  216. A. Wang and K. Cho, “Bert has a mouth, and it must speak: Bert as a markov random field language model,” in Proceedings of the Workshop on Methods for Optimizing and Evaluating Neural Language Generation, 2019, pp. 30–36.
  217. B. Yu, Y. Wang, T. Liu, H. Zhu, L. Sun, and B. Wang, “Maximal clique based non-autoregressive open information extraction,” in EMNLP, 2021, pp. 9696–9706.
  218. S. Yu, J. Tack, S. Mo, H. Kim, J. Kim, J.-W. Ha, and J. Shin, “Generating videos with dynamics-aware implicit generative adversarial networks,” in ICLR, 2021.
  219. T. Hayashi, W.-C. Huang, K. Kobayashi, and T. Toda, “Non-autoregressive sequence-to-sequence voice conversion,” in ICASSP.   IEEE, 2021, pp. 7068–7072.
  220. Z. Chen, H. Miao, and P. Zhang, “Streaming non-autoregressive model for any-to-many voice conversion,” arXiv:2206.07288, 2022.
  221. Y. Wang and Z. Zhao, “Fastlts: Non-autoregressive end-to-end unconstrained lip-to-speech synthesis,” in ACM MM, 2022, pp. 5678–5687.
  222. A. H. Liu, Y.-A. Chung, and J. Glass, “Non-autoregressive predictive coding for learning speech representations from local dependencies,” arXiv:2011.00406, 2020.
  223. F. Liu, Z. Fu, G. Li, Z. Jin, H. Liu, and Y. Hao, “Non-autoregressive model for full-line code completion,” arXiv:2204.09877, 2022.
  224. E. J. Barezi, I. Calixto, K. Cho, and P. Fung, “A study on the autoregressive and non-autoregressive multi-label learning,” arXiv:2012.01711, 2020.
  225. S. Feng, K. Xu, J. Wu, P. Wu, F. Lin, and P. Zhao, “Multi-scale attention flow for probabilistic time series forecasting,” arXiv:2205.07493, 2022.
  226. F. Huang, T. Tao, H. Zhou, L. Li, and M. Huang, “On the learning of non-autoregressive transformers,” in ICML.   PMLR, 2022, pp. 9356–9376.
  227. Y. Oka, K. Sudoh, and S. Nakamura, “Using perturbed length-aware positional encoding for non-autoregressive neural machine translation,” arXiv:2107.13689, 2021.
  228. X. Kong, Z. Zhang, and E. Hovy, “Incorporating a local translation mechanism into non-autoregressive translation,” in EMNLP, 2020, pp. 1067–1073.
  229. C. Shao, Y. Feng, J. Zhang, F. Meng, X. Chen, and J. Zhou, “Retrieving sequential information for non-autoregressive neural machine translation,” in ACL, 2019, pp. 3013–3024.
  230. L. Qin, F. Wei, T. Xie, X. Xu, W. Che, and T. Liu, “Gl-gin: Fast and accurate non-autoregressive model for joint multiple intent detection and slot filling,” in ACL-IJCNLP, 2021, pp. 178–188.
  231. D. Wu, L. Ding, F. Lu, and J. Xie, “Slotrefine: A fast non-autoregressive model for joint intent detection and slot filling,” in EMNLP, 2020, pp. 1932–1937.
  232. H. Le, R. Socher, and S. C. Hoi, “Non-autoregressive dialog state tracking,” in ICLR, 2021.
  233. T. Komatsu, “Non-autoregressive asr with self-conditioned folded encoders,” in ICASSP.   IEEE, 2022, pp. 7427–7431.
  234. T. Wang, Y. Fujita, X. Chang, and S. Watanabe, “Streaming end-to-end asr based on blockwise non-autoregressive models,” arXiv:2107.09428, 2021.
  235. S. Beliaev and B. Ginsburg, “Talknet 2: Non-autoregressive depth-wise separable convolutional model for speech synthesis with explicit pitch and duration prediction,” arXiv:2104.08189, 2021.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Yisheng Xiao (3 papers)
  2. Lijun Wu (113 papers)
  3. Junliang Guo (39 papers)
  4. Juntao Li (89 papers)
  5. Min Zhang (630 papers)
  6. Tao Qin (201 papers)
  7. Tie-Yan Liu (242 papers)
Citations (75)
View on Semantic Scholar
Github Logo Streamline Icon: https://streamlinehq.com

GitHub

  1. GitHub - LitterBrother-Xiao/Overview-of-Non-autoregressive-Applications (163 stars)
Youtube Logo Streamline Icon: https://streamlinehq.com

YouTube