• A collection of papers and resources related to Large Language Models has been compiled and organized.
  • The survey covers various open-source and closed-source models, commonly used corpora, deep learning frameworks, and pre-training data collection.

Key terms:

  • Open-source Models: Language models that are publicly available for researchers and developers to use and modify.
  • Closed-source Models: Language models that are not publicly available and are typically developed by private companies.
  • Deep Learning Frameworks: Software libraries that provide tools for building, training, and evaluating deep learning models.
  • Pre-training Data Collection: The process of gathering and organizing data for training large language models.


ChatGPT Research GPT-4 Tools GPT-3 LLaMA Large Language Models Prompt Engineering Alignment LaMDA