• MEGA is the first comprehensive benchmarking of generative Large Language Models (LLMs) that evaluates their performance on 8 diverse tasks and 33 typologically diverse languages.
  • The study compares generative LLMs to state-of-the-art non-autoregressive models and presents a framework for evaluating generative LLMs in the multilingual setting.

Key terms:

  • Generative AI: Artificial intelligence models capable of generating content such as text, images, or music
  • MEGA: A comprehensive benchmarking for evaluating generative LLMs on diverse tasks and languages
  • Non-autoregressive models: AI models that do not generate output sequentially but instead generate all elements simultaneously
  • Multilingual setting: Evaluating AI models on tasks and languages other than English


Research Large Language Models Generative AI Natural Language Processing Multilingual Setting Multilingual State of the Art Language evaluation Language capabilities Multilingual Evaluation